2025-09-01 Mon 02:10:17:421.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:421.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692617421565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:421.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:422.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:422.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692617422103 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40145 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:422.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:422.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:17:422.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:17:422.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:422.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692617422380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:422.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:18:385.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:18:385.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:18:385.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:18:385.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692618385492 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:18:386.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:18:386.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:18:386.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692618385681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38219 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:18:386.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:18:386.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:18:386.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692618385996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:18:386.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:18:612.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:18:612.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692618612535 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:18:612.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:18:612.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:18:612.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692618612709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42101 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:18:612.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:18:613.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:18:613.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:18:613.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:18:613.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692618613111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:18:613.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:037.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:037.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692619037072 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:037.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:037.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:037.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692619037294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:037.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:037.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:19:037.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:19:037.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:037.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692619037853 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:037.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:037.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:037.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692619037868 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:037.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:037.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:037.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692619037940 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:19:037.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:19:038.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:038.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692619038010 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:19:038.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:19:038.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:038.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692619038645 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:038.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:038.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:038.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:19:038.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692619038701 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:19:039.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:19:090.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:090.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692619090821 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:090.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:091.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:091.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692619091278 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34835 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:091.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:091.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:19:091.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:19:091.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:091.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692619091837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:091.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:248.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:52086 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:19:259.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:52092 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:19:558.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:558.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692619558287 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:558.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:558.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:558.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692619558453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33365 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:558.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:558.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:19:558.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:19:558.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:558.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692619558854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:558.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:558.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:558.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692619558871 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:558.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:558.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:558.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692619558924 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:19:559.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:19:559.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:559.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692619559009 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:19:559.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:19:559.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:559.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692619559026 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:19:559.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:19:559.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:559.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692619559559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:559.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:559.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:559.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692619559597 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:19:559.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:19:559.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:19:609.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:609.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692619609806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:609.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:610.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:610.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692619610063 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32997 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:610.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:19:610.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:19:610.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:19:610.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:19:610.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692619610544 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:19:610.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:20:145.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:20:145.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692620144994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:20:145.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:20:145.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:20:145.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692620145188 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44833 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:20:145.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:20:145.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:20:145.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:20:145.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:20:145.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692620145820 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:20:145.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:20:420.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:20:420.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692620420153 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:20:420.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:20:420.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:20:420.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692620420626 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42595 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:20:420.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:20:420.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:20:421.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:20:421.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:20:421.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692620421059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:20:421.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:20:618.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:20:618.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692620618673 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:20:618.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:20:619.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:20:619.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692620619154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39151 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:20:619.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:20:619.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:20:619.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:20:619.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:20:619.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692620619722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:20:619.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:054.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:054.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692622053960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:054.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:054.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:054.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692622054391 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40703 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:054.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:054.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:22:054.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:22:054.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:054.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692622054904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:054.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:321.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:321.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692622321306 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:321.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:322.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:322.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692622321995 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43579 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:322.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:322.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:22:322.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:22:322.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:322.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692622322328 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:322.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:322.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:322.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692622322333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:322.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:322.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:322.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692622322386 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:22:322.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:22:322.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:322.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692622322477 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:22:322.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:22:322.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:322.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692622322495 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:22:322.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:22:323.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:323.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692622323028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:323.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:323.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:323.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692622323072 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:22:323.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:22:323.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:22:376.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:376.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692622376397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:376.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:376.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:376.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692622376629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41211 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:376.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:376.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:22:376.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:22:377.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:377.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692622377004 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:377.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:437.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:437.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692622437899 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:438.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:438.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:438.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692622438312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38359 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:438.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:22:438.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:22:438.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:22:438.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:22:438.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692622438815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:22:438.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:23:331.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:23:331.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692623331211 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:23:331.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:23:331.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:23:331.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692623331397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34915 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:23:331.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:23:331.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:23:331.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:23:331.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:23:331.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692623331688 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:23:331.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:24:489.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:489.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692624489193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:24:489.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:24:489.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:489.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692624489624 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:24:489.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:24:489.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:24:489.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:24:490.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:490.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692624489924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:24:490.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:24:490.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:490.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692624489939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:24:490.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:24:490.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:490.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692624490002 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:24:490.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:24:490.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:490.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692624490059 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:24:490.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:24:490.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:490.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692624490593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:24:490.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:24:491.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:491.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:24:491.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692624491259 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:24:491.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:24:536.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:536.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692624536808 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:24:536.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:24:536.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:537.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692624536985 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41923 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:24:537.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:24:537.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:24:537.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:24:537.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:537.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692624537449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:24:537.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:401.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:401.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692625401840 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:401.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:402.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:402.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692625402439 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44379 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:402.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:402.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:25:402.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:25:402.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:402.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692625402761 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:402.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:581.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:581.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692625581601 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:581.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:582.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:582.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692625582186 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34081 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:582.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:582.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:25:582.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:25:582.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:582.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692625582603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:582.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:727.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:727.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692625727039 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:727.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:727.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:727.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692625727239 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38673 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:727.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:727.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:25:727.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:25:727.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:727.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692625727618 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:727.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:727.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:727.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692625727625 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:727.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:727.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:727.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692625727724 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:25:727.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:25:727.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:727.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692625727783 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:25:727.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:25:728.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:728.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692625728312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:728.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:728.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:728.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:25:728.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692625728351 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:25:728.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:25:774.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:774.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692625774539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:774.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:775.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:775.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692625775036 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35165 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:775.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:25:775.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:25:775.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:25:775.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:775.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692625775320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:25:775.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:26:507.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:507.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692626506980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:26:507.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:26:507.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:507.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692626507174 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43333 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:26:507.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:26:507.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:26:507.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:26:507.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:507.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692626507685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:26:507.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:26:507.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:507.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692626507690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:26:507.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:26:507.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:507.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692626507747 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:26:507.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:26:507.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:507.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692626507838 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:26:507.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:26:507.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:507.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692626507857 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:26:507.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:26:508.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:508.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692626508425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:26:508.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:26:508.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:508.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692626508471 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:26:508.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:26:508.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:26:557.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:557.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692626557737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:26:557.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:26:558.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:558.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692626558298 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37161 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:26:558.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:26:558.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:26:558.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:26:558.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:558.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692626558664 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:26:558.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:26:810.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:810.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692626810485 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:26:810.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:26:810.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:810.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692626810688 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43151 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:26:810.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:26:810.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:26:810.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:26:810.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:810.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692626810959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:26:810.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:27:580.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:27:580.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692627580792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:27:580.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:27:580.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:27:581.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692627580976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39169 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:27:581.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:27:582.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:27:582.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:27:582.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:27:582.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692627582607 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:27:582.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:27:593.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:27:594.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692627593741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:27:594.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:27:594.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:27:594.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692627593870 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:27:594.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:27:594.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:27:594.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:27:594.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:27:594.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692627594266 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:27:594.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:28:791.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:28:791.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692628791090 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:28:791.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:28:791.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:28:791.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692628791280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39241 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:28:791.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:28:791.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:28:791.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:28:791.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:28:791.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692628791643 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:28:791.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:29:538.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:538.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692629538428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:29:538.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:29:538.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:538.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692629538603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41579 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:29:538.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:29:538.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:29:538.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:29:538.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:539.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692629538982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:29:539.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:29:542.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:542.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692629542861 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:29:542.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:29:542.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:543.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692629542990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:29:543.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:29:543.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:29:543.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:29:543.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:543.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692629543255 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:29:543.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:29:543.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:543.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692629543261 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:29:543.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:29:543.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:543.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692629543358 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:29:543.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:29:543.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:543.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692629543445 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:29:543.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:29:543.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:543.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692629543463 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:29:543.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:29:544.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:544.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692629544012 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:29:544.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:29:544.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:544.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:29:544.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692629544066 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:29:544.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:29:595.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:595.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692629595126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:29:595.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:29:595.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:595.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692629595593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42383 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:29:595.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:29:596.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:29:596.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:29:596.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:29:596.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692629596126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:29:596.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:30:640.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:30:640.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692630640302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:30:640.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:30:640.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:30:640.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692630640518 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:30:640.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:30:640.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:30:640.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:30:641.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:30:641.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692630641021 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:30:641.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:31:434.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:434.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692631434790 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:31:434.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:31:434.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:435.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692631434987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36739 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:31:435.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:31:435.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:31:435.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:31:435.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:435.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692631435513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:31:435.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:31:435.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:435.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692631435521 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:31:435.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:31:435.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:435.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692631435589 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:31:435.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:31:435.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:435.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692631435652 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:31:435.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:31:436.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:436.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692631436227 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:31:436.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:31:436.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:436.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692631436268 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:31:436.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:31:436.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:31:484.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:484.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692631484560 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:31:484.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:31:485.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:485.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692631485112 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46711 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:31:485.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:31:485.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:31:485.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:31:485.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:485.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692631485462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:31:485.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:31:833.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:833.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692631833037 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 5353 dequeues = 5353 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 5417 dequeues = 5417 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692602860238 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692631485469 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:10:31:833.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:31:833.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:833.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692631833156 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1069 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 97 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1215 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 292 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4119 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1177 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2687, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5353, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:31:833.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:31:841.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:841.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692631841047 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.25, ] ] ] 2025-09-01 Mon 02:10:31:841.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:10:31:841.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:841.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692631841089 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1547 in_bytes = 188863 out_tpkts = 1614 out_bytes = 90187 raw_vhost_stats= [ name = vhost0 in_pkts = 1547 in_bytes = 188863 out_pkts = 1614 out_bytes = 90187 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 6.77084e-15 total_out_bandwidth_utilization = 1.59595e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1249, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 2944, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1444 in_bytes = 83047 out_pkts = 1550 out_bytes = 188989 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:10:31:841.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:10:31:841.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:31:841.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692631841122 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:10:31:841.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:10:32:385.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:385.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692632385667 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:385.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:385.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:385.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692632385877 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:385.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:386.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:32:386.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:32:386.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:386.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692632386342 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:386.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:477.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:477.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692632477191 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:477.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:477.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:477.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692632477806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40535 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:477.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:478.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:32:478.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:32:478.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:478.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692632478315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:478.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:787.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:790.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692632787830 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:790.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:788.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:32:790.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:790.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692632788030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35925 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:790.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:790.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:32:790.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:790.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692632790295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:790.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:790.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:790.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692632790301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:790.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:790.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:790.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692632790355 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:32:790.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:32:790.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:790.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692632790405 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:32:790.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:32:791.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:791.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692632790999 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:791.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:791.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:791.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692632791036 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:32:791.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:32:791.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:32:839.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:839.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692632839203 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:839.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:839.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:839.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692632839756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40805 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:839.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:840.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:32:840.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:32:840.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:840.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692632840224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:32:840.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:32:860.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:10:32:860.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:10:32:860.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:10:32:860.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692632860469 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:32:860.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:32:860.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:10:32:860.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:10:32:860.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:10:32:861.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692632860758 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1072 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 99 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1219 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 293 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4133 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1181 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2696, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5375, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:32:861.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:32:861.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:10:32:861.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:10:32:861.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:32:861.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Idle 2025-09-01 Mon 02:10:32:861.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:861.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692632861217 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1073 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1219 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 293 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4133 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1181 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2696, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5377, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:32:861.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:32:861.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:861.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692632861246 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:32:861.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:32:861.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:10:32:861.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:32:861.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692632861347 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1073 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1220 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 293 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4133 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1181 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2697, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5377, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:32:861.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:33:794.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:794.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692633794380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:33:794.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:33:795.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:795.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692633794991 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43769 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:33:795.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:33:795.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:33:795.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:33:795.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:795.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692633795465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:33:795.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:33:823.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:823.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692633823185 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:33:823.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:33:823.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:823.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692633823704 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34957 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:33:823.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:33:823.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:33:824.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:33:824.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:824.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692633824154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:33:824.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:33:824.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:824.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692633824161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:33:824.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:33:824.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:824.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692633824216 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:33:824.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:33:824.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:824.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692633824303 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:33:824.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:33:824.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:824.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692633824353 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:33:824.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:33:825.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:825.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692633824988 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:33:825.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:33:825.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:825.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:33:825.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692633825031 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:33:825.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:33:870.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:870.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692633870206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:33:870.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:33:870.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:870.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692633870716 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:33:870.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:33:871.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:33:871.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:33:871.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:33:871.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692633871306 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:33:871.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:34:240.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:34:240.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692634240488 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:34:240.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:34:241.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:34:241.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692634241098 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42163 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:34:241.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:34:241.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:34:241.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:34:241.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:34:241.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692634241492 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:34:241.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:34:807.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:34:807.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692634807441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:34:807.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:34:807.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:34:807.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692634807897 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45665 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:34:807.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:34:808.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:34:808.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:34:808.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:34:808.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692634808374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:34:808.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:35:901.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:35:901.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692635901561 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:35:901.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:35:902.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:35:902.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692635902031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33263 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:35:902.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:35:902.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:35:902.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:35:902.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:35:902.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692635902514 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:35:902.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:36:294.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:294.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692636294480 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:36:294.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:36:294.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:294.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692636294689 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33679 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:36:294.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:36:295.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:36:295.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:36:295.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:295.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692636295212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:36:295.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:36:295.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:295.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692636295220 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:36:295.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:36:295.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:295.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692636295298 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:36:295.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:36:295.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:295.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692636295403 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:36:295.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:36:295.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:295.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692636295421 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:36:295.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:36:296.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:296.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692636295967 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:36:296.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:36:296.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:296.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692636296024 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:36:296.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:36:296.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:36:343.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:343.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692636343248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:36:343.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:36:343.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:343.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692636343712 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42939 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:36:343.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:36:344.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:36:344.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:36:344.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:344.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692636344167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:36:344.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:36:788.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:788.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692636788535 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:36:788.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:36:789.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:789.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692636789156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40669 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:36:789.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:36:789.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:36:789.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:36:789.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:36:789.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692636789412 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:36:789.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:37:325.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:37:325.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692637325315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:37:325.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:37:325.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:37:325.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692637325779 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41965 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:37:325.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:37:326.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:37:326.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:37:326.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:37:326.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692637326280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:37:326.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:38:273.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:38:273.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692638273672 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:38:273.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:38:274.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:38:274.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692638274268 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45305 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:38:274.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:38:274.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:38:274.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:38:274.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:38:275.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:38:275.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692638274629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:38:275.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:38:275.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:38:275.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692638274636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:38:275.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:38:275.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:38:275.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692638274694 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:38:275.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:38:275.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:38:275.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692638274741 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:38:275.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:38:275.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:38:275.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692638275338 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:38:275.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:38:275.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:38:275.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692638275382 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:38:275.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:38:324.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:38:324.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692638324527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:38:324.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:38:325.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:38:325.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692638325043 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37047 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:38:325.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:38:325.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:38:325.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:38:325.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:38:325.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692638325338 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:38:325.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:360.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:360.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692639360534 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:360.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:360.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:360.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:39:361.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:39:361.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692639360763 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36951 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:361.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:361.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:361.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692639361062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:361.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:414.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:414.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692639414385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:414.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:414.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:415.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692639414964 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45359 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:415.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:415.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:39:415.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:39:415.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:415.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692639415346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:415.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:538.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:538.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692639538682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:538.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:539.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:539.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692639539292 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32819 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:539.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:539.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:39:540.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:39:540.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:540.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692639540057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:540.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:540.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:540.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692639540062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:540.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:540.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:540.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692639540115 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:39:540.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:39:540.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:540.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692639540181 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:39:540.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:39:540.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:540.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692639540709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:540.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:540.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:540.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692639540737 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:39:540.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:39:540.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:39:589.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:589.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692639589015 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:589.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:589.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:589.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692639589226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39045 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:589.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:39:589.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:39:589.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:39:589.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:39:589.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692639589619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:39:589.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:569.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:569.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692640569617 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:569.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:570.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:570.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692640570110 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:570.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:570.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:40:570.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:40:570.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:570.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692640570498 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:570.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:570.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:570.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692640570505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:570.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:570.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:570.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692640570586 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:40:570.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:40:570.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:570.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692640570687 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:40:570.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:40:570.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:570.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692640570705 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:40:570.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:40:571.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:571.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692640571260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:571.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:571.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:571.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:40:571.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692640571329 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:40:571.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:40:606.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:607.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692640606733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:607.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:607.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:607.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692640606903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36939 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:607.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:607.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:40:607.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:40:607.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:607.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692640607319 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:607.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:620.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:620.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692640620433 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:620.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:620.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:621.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692640620959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43463 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:621.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:621.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:40:621.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:40:621.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:621.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692640621357 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:621.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:41:215.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:41:215.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692641215224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:41:215.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:41:215.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:41:215.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692641215429 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45637 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:41:215.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:41:215.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:41:215.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:41:215.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:41:215.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692641215788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:41:215.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:41:611.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:41:611.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692641611466 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:41:611.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:41:611.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:41:611.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692641611885 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33921 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:41:611.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:41:612.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:41:612.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:41:612.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:41:612.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692641612343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:41:612.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:42:587.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:42:587.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692642587542 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:42:587.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:42:588.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:42:588.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692642588031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45197 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:42:588.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:42:588.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:42:588.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:42:588.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:42:588.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692642588470 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:42:588.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:43:574.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:574.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692643574512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:43:574.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:43:575.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:575.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692643575013 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41645 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:43:575.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:43:575.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:43:575.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:43:575.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:575.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692643575477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:43:575.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:43:699.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:699.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692643699522 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:43:699.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:43:700.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:700.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692643700056 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:43:700.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:43:700.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:43:700.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:43:700.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:700.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692643700504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:43:700.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:43:700.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:700.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692643700512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:43:700.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:43:700.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:700.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692643700594 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:43:700.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:43:700.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:700.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692643700694 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:43:700.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:43:700.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:700.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692643700713 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:43:700.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:43:701.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:701.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692643701332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:43:701.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:43:701.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:701.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692643701371 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:43:701.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:43:701.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:43:749.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:749.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692643749563 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:43:749.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:43:749.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:749.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692643749752 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33149 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:43:749.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:43:749.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:43:750.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:43:750.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:43:750.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692643750053 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:43:750.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:44:677.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:44:677.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692644677187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:44:677.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:44:677.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:44:677.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692644677742 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35069 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:44:677.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:44:677.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:44:677.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:44:678.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:44:678.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692644678005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:44:678.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:45:392.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:45:392.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692645391983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:45:392.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:45:392.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:45:392.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692645392269 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39281 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:45:392.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:45:392.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:45:392.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:45:392.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:45:392.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692645392615 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:45:392.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:45:392.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:45:392.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692645392620 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:45:392.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:45:392.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:45:392.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692645392707 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:45:392.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:45:392.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:45:392.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692645392755 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:45:392.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:45:393.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:45:393.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:45:393.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692645393296 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:45:393.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:45:393.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:45:393.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692645393373 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:45:393.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:45:446.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:45:446.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692645446498 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:45:446.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:45:447.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:45:447.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692645447156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46573 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:45:447.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:45:447.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:45:447.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:45:447.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:45:447.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692645447585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:45:447.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:455.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:456.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692646455780 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:456.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:456.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:456.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692646455962 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44011 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:456.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:456.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:46:456.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:46:456.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:456.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692646456367 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:456.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:568.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:568.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692646568122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:568.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:568.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:568.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692646568713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37783 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:568.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:568.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:46:569.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:46:569.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:569.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692646569070 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:569.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:584.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:584.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692646584637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:584.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:585.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:585.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692646585122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36935 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:585.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:585.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:46:585.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:46:585.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:585.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692646585590 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:585.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:585.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:585.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692646585596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:585.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:585.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:585.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692646585669 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:46:585.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:46:585.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:585.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692646585727 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:46:585.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:46:586.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:586.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:46:586.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692646586277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:586.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:586.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:586.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692646586321 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:46:586.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:46:633.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:633.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692646633478 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:633.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:634.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:634.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692646634050 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34681 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:634.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:46:634.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:46:634.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:46:634.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:46:634.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692646634316 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:46:634.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:175.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:175.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692647175620 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:175.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:176.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:176.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692647176077 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41613 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:176.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:176.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:47:176.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:47:176.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:176.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692647176533 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:176.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:176.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:176.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692647176540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:176.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:176.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:176.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692647176602 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:47:176.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:47:177.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:177.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692647177158 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:47:177.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:47:177.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:177.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692647177184 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:47:177.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:47:177.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:177.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692647177867 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:177.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:177.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:177.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:47:177.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692647177929 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:47:178.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:47:230.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:230.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692647230036 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:230.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:230.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:230.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692647230591 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39511 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:230.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:230.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:47:230.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:47:230.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:230.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692647230904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:230.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:642.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:642.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692647642472 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:642.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:643.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:643.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692647643035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39691 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:643.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:643.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:47:643.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:47:643.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:643.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692647643324 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:643.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:48:275.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:48:275.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692648275044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:48:276.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:48:275.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:48:276.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:48:277.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692648275565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38419 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:48:277.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:48:277.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:48:277.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:48:277.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692648277090 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:48:277.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:48:292.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:48:292.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692648292484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:48:292.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:48:292.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:48:292.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692648292650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39377 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:48:292.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:48:294.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:48:294.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:48:294.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:48:294.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692648294499 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:48:294.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:49:443.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:49:443.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692649443458 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:49:443.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:49:443.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:49:443.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692649443904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40421 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:49:443.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:49:444.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:49:444.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:49:444.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:49:444.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692649444454 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:49:444.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:50:257.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:257.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692650257262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:50:257.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:50:257.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:257.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692650257726 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39451 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:50:257.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:50:258.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:50:258.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:50:258.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:258.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692650258266 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:50:258.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:50:709.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:709.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692650709279 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:50:709.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:50:709.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:709.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692650709858 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42369 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:50:709.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:50:710.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:50:710.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:50:710.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:710.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692650710099 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:50:710.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:50:710.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:710.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692650710105 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:50:710.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:50:710.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:710.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692650710187 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:50:710.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:50:710.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:710.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692650710436 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:50:710.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:50:710.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:710.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692650710456 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:50:710.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:50:711.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:711.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692650710994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:50:711.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:50:711.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:711.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692650711036 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:50:711.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:50:711.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:50:756.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:756.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692650756225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:50:756.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:50:756.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:756.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692650756793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41701 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:50:756.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:50:757.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:50:757.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:50:757.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:50:757.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692650757234 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:50:757.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:51:666.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:51:666.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692651666474 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:51:666.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:51:666.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:51:666.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692651666713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46177 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:51:666.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:51:667.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:51:667.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:51:667.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:51:667.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692651667176 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:51:667.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:52:038.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:52:038.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692652038655 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:52:038.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:52:038.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:52:038.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692652038831 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38183 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:52:038.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:52:039.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:52:039.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:52:039.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:52:040.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692652039254 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:52:040.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:52:040.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:52:040.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692652039265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:52:040.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:52:040.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:52:040.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692652039330 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:52:040.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:52:040.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:52:040.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692652039383 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:52:040.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:52:040.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:52:040.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692652039923 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:52:040.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:52:040.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:52:040.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692652039963 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:52:040.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:52:039.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:52:087.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:52:087.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692652087021 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:52:087.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:52:087.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:52:087.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692652087552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36991 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:52:087.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:52:087.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:52:087.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:52:087.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:52:087.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692652087945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:52:087.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:080.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:080.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692653080637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:080.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:080.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:080.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692653080837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43843 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:080.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:081.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:53:081.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:53:081.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:081.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692653081168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:081.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:081.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:081.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692653081174 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:081.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:081.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:081.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692653081252 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:53:081.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:53:081.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:081.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692653081298 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:53:081.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:53:082.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:082.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692653082023 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:082.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:082.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:082.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:53:082.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692653082069 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:53:082.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:53:133.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:133.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692653133230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:133.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:133.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:133.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692653133703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41853 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:133.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:134.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:53:134.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:53:134.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:134.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692653134269 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:134.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:159.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:159.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692653159133 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:159.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:159.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:159.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692653159567 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33759 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:159.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:159.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:53:160.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:53:160.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:160.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692653160054 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:160.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:665.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:665.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692653665379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:665.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:666.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:666.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692653666003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36871 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:666.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:53:666.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:53:666.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:53:666.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:53:666.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692653666509 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:53:666.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:54:038.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:038.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692654038427 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:54:038.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:54:038.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:038.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692654038942 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43653 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:54:038.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:54:039.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:54:039.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:54:039.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:039.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692654039351 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:54:039.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:54:039.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:039.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692654039357 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:54:039.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:54:039.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:039.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692654039433 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:54:039.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:54:039.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:039.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692654039533 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:54:039.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:54:039.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:039.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692654039552 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:54:039.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:54:040.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:040.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692654040102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:54:040.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:54:040.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:040.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692654040129 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:54:040.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:54:040.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:54:043.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:043.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692654043352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:54:043.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:54:043.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:043.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692654043457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37679 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:54:043.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:54:043.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:54:043.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:54:043.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:043.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692654043825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:54:043.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:54:089.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:089.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692654089277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:54:089.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:54:089.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:089.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692654089464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35251 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:54:089.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:54:089.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:54:089.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:54:089.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:089.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692654089795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:54:089.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:55:050.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:050.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692655050272 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:55:050.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:55:050.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:050.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692655050740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35393 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:55:050.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:55:051.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:55:051.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:55:051.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:051.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692655051222 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:55:051.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:55:160.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:161.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692655160914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:55:161.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:55:161.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:161.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692655161502 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46177 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:55:161.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:55:161.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:55:161.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:55:161.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:161.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692655161792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:55:161.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:55:844.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:844.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692655844010 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:55:844.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:55:844.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:844.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692655844232 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44301 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:55:844.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:55:844.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:55:844.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:55:844.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:844.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692655844716 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:55:844.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:123.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:124.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692657123948 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:124.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:124.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:124.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692657124541 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36441 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:124.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:124.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:57:124.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:57:124.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:124.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692657124933 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:124.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:626.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:626.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692657626569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:626.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:626.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:626.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692657626739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39387 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:626.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:627.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:57:627.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:57:627.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:627.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692657627619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:627.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:627.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:627.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692657627631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:627.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:627.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:627.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692657627694 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:57:627.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:57:627.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:627.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692657627779 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:57:627.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:57:627.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:627.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692657627800 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:57:627.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:57:628.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:628.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692657628355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:628.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:628.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:628.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692657628399 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:57:628.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:57:628.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:57:676.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:676.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692657676592 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:676.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:677.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:677.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692657677192 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35327 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:677.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:677.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:57:677.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:57:677.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:677.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692657677624 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:677.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:58:622.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:58:623.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:58:624.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692658622801 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:58:624.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:58:624.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:58:624.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692658622976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36671 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:58:624.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:58:624.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:58:624.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:58:624.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692658624597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:58:624.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:59:047.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:59:047.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692659047388 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:59:047.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:59:048.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:59:048.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692659048040 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32773 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:59:048.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:59:048.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:59:048.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:59:048.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:59:049.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:59:049.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692659048436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:59:049.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:59:049.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:59:049.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692659048444 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:59:049.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:59:049.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:59:049.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692659048506 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:59:049.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:59:049.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:59:049.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692659048560 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:59:049.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:59:049.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:59:049.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692659049082 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:59:049.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:59:049.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:59:049.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692659049121 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:59:049.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:59:097.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:59:097.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692659097248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:59:097.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:59:097.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:59:097.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692659097449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35529 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:59:097.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:59:097.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:59:097.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:59:097.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:59:097.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692659097783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:59:097.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:016.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:016.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692660015983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:016.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:016.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:016.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692660016724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40071 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:016.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:017.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:00:017.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:00:017.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:017.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692660017165 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:017.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:092.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:092.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692660092828 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:092.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:093.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:093.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692660093319 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46123 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:093.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:093.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:00:093.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:00:093.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:093.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692660093828 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:093.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:093.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:093.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692660093835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:093.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:093.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:093.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692660093894 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:00:093.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:00:093.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:093.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692660093938 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:00:093.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:00:094.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:094.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692660094534 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:094.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:094.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:094.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692660094588 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:00:094.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:00:094.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:00:139.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:139.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692660139831 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:139.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:140.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:140.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692660140272 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43215 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:140.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:140.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:00:140.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:00:140.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:140.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692660140762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:140.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:425.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:425.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692660424836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:425.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:426.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:426.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692660426216 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44789 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:426.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:00:426.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:00:426.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:00:426.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:00:426.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692660426811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:00:426.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:166.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:167.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692661166921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:167.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:167.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:167.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692661167514 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36951 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:167.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:167.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:01:167.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:01:167.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:167.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692661167911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:167.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:265.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:265.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692661265084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:265.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:265.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:265.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692661265640 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35477 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:265.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:265.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:01:265.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:01:266.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:266.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692661266033 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:266.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:266.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:266.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692661266041 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:266.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:266.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:266.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692661266105 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:01:266.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:01:266.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:266.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692661266490 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:01:266.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:01:266.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:266.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692661266516 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:01:266.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:01:267.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:267.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692661267060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:267.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:267.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:267.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692661267103 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:01:267.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:01:267.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:01:312.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:312.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692661312278 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:312.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:312.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:312.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692661312836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46851 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:312.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:313.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:01:313.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:01:313.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:313.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692661313301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:313.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:833.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:833.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692661833252 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 5675 dequeues = 5675 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 5742 dequeues = 5742 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692632861302 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692661313307 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:11:01:833.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:01:833.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:833.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692661833367 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1134 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 102 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1287 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 310 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4368 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1248 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2848, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5675, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:01:833.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:01:841.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:841.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692661841346 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:11:01:841.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:11:01:841.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:841.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692661841409 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1375572704 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.155 five_min_avg = 0.085 fifteen_min_avg = 0.12 ] cpu_share = 1.11667 ] ] 2025-09-01 Mon 02:11:01:841.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:11:01:841.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:841.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692661841417 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.11667 used_sys_mem = 2854240503 one_min_cpuload = 0.155 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:11:01:841.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:11:01:841.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:841.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692661841452 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1627 in_bytes = 194523 out_tpkts = 1694 out_bytes = 94427 exception_packets = 57 exception_packets_dropped = 13 exception_packets_allowed = 44 raw_vhost_stats= [ name = vhost0 in_pkts = 1627 in_bytes = 194523 out_pkts = 1694 out_bytes = 94427 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1521 in_bytes = 87161 out_pkts = 1630 out_bytes = 194649 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:11:01:841.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:11:01:841.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:841.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692661841485 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:11:01:841.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:11:01:943.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:943.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692661943304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:943.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:943.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:943.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692661943479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42579 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:943.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:01:943.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:01:943.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:01:943.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:01:943.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692661943891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:01:943.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:02:267.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:02:267.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692662267441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:02:267.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:02:267.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:02:267.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692662267913 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46643 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:02:267.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:02:268.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:02:268.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:02:268.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:02:268.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692662268400 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:02:268.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:02:861.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:11:02:862.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:11:02:862.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:11:02:862.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692662861496 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:02:862.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:02:862.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:11:02:862.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:11:02:862.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:11:02:862.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692662862171 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1134 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 104 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1289 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 310 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4374 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1250 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2852, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5688, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:02:862.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:02:862.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:11:02:862.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:11:02:862.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:02:862.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Idle 2025-09-01 Mon 02:11:02:862.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:02:862.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692662862561 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1135 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 105 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1289 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 310 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4374 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1250 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2852, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5690, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:02:862.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:02:862.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:02:862.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692662862589 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:02:862.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:02:862.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:11:02:862.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:02:863.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692662862692 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1135 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 105 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1290 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 310 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4374 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1250 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2853, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5690, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:02:863.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:03:058.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:03:058.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692663058092 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:03:058.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:03:058.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:03:058.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692663058333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41147 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:03:058.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:03:058.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:03:058.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:03:058.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:03:058.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692663058755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:03:058.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:248.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:249.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:04:250.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692664248527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:04:250.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:250.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:250.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692664249117 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39369 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:04:250.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:250.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:04:250.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:250.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692664250536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:04:250.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:603.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:603.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692664602987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:04:603.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:603.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:603.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692664603581 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39605 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:04:603.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:603.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:04:603.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:04:603.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:604.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692664603946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:04:604.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:604.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:604.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692664603953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:04:604.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:604.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:604.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692664604016 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:04:604.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:04:604.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:604.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692664604245 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:04:604.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:04:604.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:604.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692664604264 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:04:604.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:04:604.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:604.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692664604790 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:04:604.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:604.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:604.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692664604848 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:04:604.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:04:604.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:04:652.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:652.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692664651992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:04:652.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:652.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:652.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692664652592 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44603 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:04:652.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:652.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:04:652.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:04:652.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:04:652.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692664652919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:04:652.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:05:580.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:580.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692665580077 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:05:580.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:05:580.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:580.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692665580662 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45541 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:05:580.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:05:580.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:05:581.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:05:581.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:581.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692665581129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:05:581.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:05:796.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:796.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692665796140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:05:796.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:05:796.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:796.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692665796371 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36659 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:05:796.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:05:796.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:05:796.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:05:796.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:797.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692665796753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:05:797.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:05:797.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:797.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692665796773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:05:797.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:05:797.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:797.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692665796844 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:05:797.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:05:797.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:797.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692665796895 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:05:797.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:05:797.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:797.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692665797433 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:05:797.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:05:797.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:797.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692665797476 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:05:797.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:05:797.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:05:842.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:842.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692665842615 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:05:842.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:05:842.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:842.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692665842792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:05:842.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:05:843.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:05:843.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:05:843.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:05:843.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692665843296 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:05:843.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:06:658.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:659.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692666658903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:06:659.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:06:659.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:659.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692666659081 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:06:659.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:06:659.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:06:659.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:06:659.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:660.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:06:660.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692666659482 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:06:660.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:06:660.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:660.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692666659489 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:06:660.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:06:660.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:660.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692666659549 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:06:660.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:06:660.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:660.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692666659602 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:06:660.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:06:660.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:660.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692666660106 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:06:660.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:06:660.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:660.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692666660156 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:06:660.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:06:707.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:707.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692666707317 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:06:707.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:06:707.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:708.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692666707945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38357 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:06:708.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:06:708.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:06:708.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:06:708.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:708.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692666708300 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:06:708.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:06:815.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:815.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692666815466 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:06:815.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:06:815.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:816.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692666815976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39481 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:06:816.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:06:816.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:06:816.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:06:816.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:06:816.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692666816536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:06:816.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:399.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:399.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692667399316 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:07:399.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:399.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:400.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692667399915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 32867 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:07:400.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:400.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:07:400.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:07:400.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:400.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692667400261 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:07:400.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:653.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:653.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692667653443 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:07:653.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:654.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:654.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692667654039 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44599 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:07:654.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:654.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:07:654.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:07:654.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:654.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692667654404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:07:654.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:994.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:994.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692667994727 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:07:994.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:995.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:995.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692667995268 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41187 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:07:995.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:995.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:07:995.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:07:995.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:995.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692667995712 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:07:995.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:995.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:995.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692667995718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:07:995.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:995.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:995.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692667995803 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:07:995.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:07:996.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:996.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692667995904 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:07:996.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:07:996.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:996.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692667995924 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:07:996.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:07:996.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:996.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692667996464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:07:996.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:07:996.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:07:996.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:07:996.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692667996491 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:07:996.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:08:048.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:08:048.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692668048648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:08:048.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:08:049.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:08:049.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692668049167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45901 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:08:049.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:08:049.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:08:049.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:08:049.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:08:049.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692668049595 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:08:049.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:08:760.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:08:760.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692668760699 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:08:760.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:08:761.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:08:761.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692668761201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37539 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:08:761.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:08:761.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:08:761.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:08:761.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:08:761.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692668761750 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:08:761.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:09:003.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:09:003.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692669003743 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:09:003.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:09:003.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:09:003.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692669003932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44507 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:09:003.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:09:004.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:09:004.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:09:004.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:09:004.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692669004324 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:09:004.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:09:508.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:09:509.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:09:510.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692669508553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:09:510.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:09:510.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:09:510.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692669509164 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40083 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:09:510.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:09:510.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:09:510.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:09:510.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692669510647 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:09:510.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:074.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:074.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692671074540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:074.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:074.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:074.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692671074774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46749 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:074.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:075.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:11:075.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:11:075.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:075.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692671075206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:075.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:180.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:180.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692671180437 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:180.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:180.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:181.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692671180934 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38045 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:181.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:181.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:11:181.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:11:181.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:181.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692671181237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:181.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:181.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:181.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692671181244 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:181.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:181.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:181.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692671181347 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:11:181.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:11:181.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:181.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692671181468 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:11:181.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:11:181.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:181.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692671181489 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:11:181.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:11:182.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:182.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692671182119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:182.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:182.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:182.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692671182180 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:11:182.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:11:182.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:11:235.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:235.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692671235400 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:235.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:235.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:235.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692671235596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34277 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:235.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:235.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:11:236.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:11:236.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:236.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692671236120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:236.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:12:289.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:289.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692672289344 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:12:289.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:12:289.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:289.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692672289729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40079 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:12:289.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:12:290.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:12:290.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:12:290.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:290.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692672290187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:12:290.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:12:829.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:830.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692672829945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:12:830.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:12:830.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:830.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692672830572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37045 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:12:830.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:12:830.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:12:830.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:12:831.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:831.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692672830993 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:12:831.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:12:831.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:831.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692672831001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:12:831.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:12:831.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:831.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692672831063 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:12:831.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:12:831.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:831.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692672831138 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:12:831.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:12:831.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:831.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692672831739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:12:831.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:12:831.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:832.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692672831781 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:12:832.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:12:831.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:12:876.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:877.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692672876961 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:12:877.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:12:877.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:877.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692672877136 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35079 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:12:877.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:12:877.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:12:877.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:12:877.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:12:877.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692672877608 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:12:877.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:13:398.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:398.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692673398812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:13:398.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:13:399.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:399.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692673399324 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43769 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:13:399.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:13:399.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:13:399.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:13:399.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:400.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692673399686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:13:400.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:13:400.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:400.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692673399693 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:13:400.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:13:400.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:400.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692673399756 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:13:400.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:13:400.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:400.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692673399804 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:13:400.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:13:400.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:400.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692673400348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:13:400.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:13:400.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:400.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692673400385 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:13:400.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:13:400.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:13:453.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:453.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692673453494 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:13:453.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:13:454.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:454.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692673454063 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35795 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:13:454.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:13:454.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:13:454.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:13:454.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:454.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692673454538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:13:454.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:13:858.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:858.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692673858736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:13:858.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:13:859.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:859.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692673859206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34913 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:13:859.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:13:859.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:13:859.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:13:859.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:13:859.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692673859662 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:13:859.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:14:378.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:14:378.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692674378371 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:14:378.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:14:378.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:14:379.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692674378944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40637 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:14:379.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:14:379.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:14:379.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:14:379.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:14:379.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692674379352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:14:379.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:14:480.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:14:480.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692674480719 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:14:480.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:14:481.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:14:481.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692674481283 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37409 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:14:481.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:14:481.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:14:481.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:14:481.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:14:481.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692674481713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:14:481.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:107.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:107.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692675107322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:107.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:107.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:107.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692675107913 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40975 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:107.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:108.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:15:108.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:15:108.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:108.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692675108320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:108.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:108.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:108.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692675108569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:108.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:108.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:108.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692675108650 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:15:108.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:15:108.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:108.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692675108853 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:15:108.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:15:108.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:108.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692675108871 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:15:108.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:15:109.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:109.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692675109902 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:109.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:15:109.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:110.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:110.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692675109948 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:15:110.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:15:155.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:155.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692675155061 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:155.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:155.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:155.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692675155262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42793 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:155.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:155.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:15:155.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:15:155.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:155.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692675155566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:155.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:713.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:713.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692675713847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:713.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:714.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:714.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692675714464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41229 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:714.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:714.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:15:714.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:15:714.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:714.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692675714841 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:714.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:16:199.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:16:199.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692676199720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:16:199.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:16:200.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:16:200.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692676200285 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:16:200.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:16:200.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:16:200.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:16:200.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:16:200.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692676200547 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:16:200.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:16:623.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:16:623.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692676623889 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:16:624.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:16:624.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:16:624.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692676624380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43401 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:16:624.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:16:624.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:16:624.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:16:624.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:16:624.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692676624775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:16:624.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:18:198.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:198.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692678198690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:18:198.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:18:199.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:199.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692678199161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43341 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:18:199.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:18:199.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:18:199.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:18:199.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:199.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692678199496 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:18:199.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:18:339.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:339.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692678339538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:18:339.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:18:339.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:339.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692678339722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33219 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:18:339.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:18:340.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:18:340.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:18:340.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:340.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692678340182 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:18:340.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:18:340.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:340.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692678340187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:18:340.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:18:340.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:340.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692678340251 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:18:340.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:18:340.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:340.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692678340347 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:18:340.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:18:340.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:340.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692678340365 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:18:340.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:18:341.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:341.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692678341333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:18:341.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:18:341.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:341.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:18:341.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692678341376 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:18:341.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:18:389.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:389.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692678389595 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:18:389.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:18:389.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:389.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692678389785 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39409 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:18:389.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:18:390.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:18:390.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:18:390.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:18:390.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692678390179 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:18:390.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:19:299.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:299.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692679299350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:19:299.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:19:299.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:299.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692679299827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32923 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:19:299.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:19:300.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:19:300.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:19:300.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:300.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692679300332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:19:300.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:19:891.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:891.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692679891028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:19:891.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:19:891.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:891.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692679891644 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45493 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:19:891.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:19:891.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:19:891.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:19:892.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:892.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:19:892.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692679892015 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:19:893.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:19:893.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:893.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692679892025 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:19:893.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:19:893.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:893.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692679892108 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:19:893.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:19:893.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:893.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692679892177 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:19:893.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:19:893.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:893.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692679892816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:19:893.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:19:893.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:893.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692679892867 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:19:893.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:19:939.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:939.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692679939029 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:19:939.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:19:939.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:939.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692679939673 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40695 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:19:939.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:19:939.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:19:939.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:19:940.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:19:940.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692679940028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:19:940.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:20:101.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:34414 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:20:113.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:34428 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:20:620.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:621.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692680620911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:20:621.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:20:621.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:621.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692680621402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33123 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:20:621.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:20:621.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:20:621.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:20:621.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:622.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692680621905 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:20:622.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:20:622.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:622.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692680621912 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:20:622.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:20:622.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:622.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692680621972 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:20:622.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:20:622.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:622.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692680622024 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:20:622.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:20:622.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:622.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692680622575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:20:622.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:20:622.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:622.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692680622616 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:20:622.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:20:622.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:20:671.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:671.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692680671781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:20:671.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:20:671.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:672.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692680671982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37597 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:20:672.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:20:672.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:20:672.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:20:672.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:672.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692680672398 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:20:672.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:20:975.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:975.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692680975205 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:20:975.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:20:975.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:975.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692680975739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36649 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:20:975.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:20:975.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:20:976.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:20:976.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:20:976.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692680976087 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:20:976.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:190.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:190.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692681190487 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:190.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:190.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:190.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692681190669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38645 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:190.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:190.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:21:191.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:21:191.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:191.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692681191072 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:191.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:603.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:603.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692681602706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:603.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:603.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:603.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692681603636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35989 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:603.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:604.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:21:604.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:21:605.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:605.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692681604986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:605.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:799.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:799.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692681799645 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:799.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:800.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:800.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692681800280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40961 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:800.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:800.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:21:800.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:21:800.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:800.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692681800580 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:800.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:800.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:800.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692681800586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:800.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:800.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:800.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692681800648 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:21:800.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:21:800.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:800.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692681800734 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:21:800.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:21:800.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:800.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692681800755 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:21:800.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:21:801.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:801.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692681801330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:801.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:801.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:801.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692681801372 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:21:801.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:21:801.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:21:846.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:846.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692681846531 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:846.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:846.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:846.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692681846762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35969 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:846.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:21:847.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:21:847.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:21:847.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:21:847.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692681847217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:21:847.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:22:810.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:22:810.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692682810360 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:22:810.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:22:810.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:22:810.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692682810816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37795 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:22:810.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:22:811.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:22:811.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:22:811.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:22:811.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692682811344 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:22:811.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:22:884.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:22:884.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692682884290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:22:884.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:22:884.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:22:884.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692682884773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41181 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:22:884.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:22:885.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:22:885.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:22:885.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:22:885.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692682885171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:22:885.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:23:747.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:23:747.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692683747524 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:23:748.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:23:748.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:23:748.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:23:748.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:23:748.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692683747750 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40307 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:23:748.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:23:748.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:23:748.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692683748213 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:23:748.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:863.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:863.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692684863173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:24:863.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:863.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:863.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692684863372 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44559 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:24:863.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:863.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:24:863.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:24:863.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:863.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692684863823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:24:863.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:863.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:863.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692684863838 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:24:863.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:863.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:863.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692684863906 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:24:863.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:24:864.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:864.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692684864035 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:24:864.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:24:864.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:864.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692684864104 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:24:864.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:24:864.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:864.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692684864660 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:24:864.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:864.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:864.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692684864702 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:24:864.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:24:864.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:24:909.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:909.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692684909818 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:24:909.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:910.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:910.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692684910237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37973 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:24:910.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:910.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:24:910.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:24:910.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:910.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692684910532 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:24:910.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:935.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:935.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692684935464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:24:935.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:935.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:935.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692684935834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46381 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:24:935.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:936.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:24:936.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:24:936.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:936.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692684936360 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:24:936.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:25:963.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:963.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692685963653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:25:963.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:25:964.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:964.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692685964197 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45239 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:25:964.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:25:964.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:25:964.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:25:964.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:964.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692685964559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:25:964.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:26:989.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:26:989.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692686989339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:26:989.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:26:989.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:26:990.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692686989927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41527 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:26:990.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:26:990.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:26:990.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:26:990.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:26:990.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692686990225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:26:990.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:26:990.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:26:990.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692686990230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:26:990.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:26:990.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:26:990.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692686990314 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:26:990.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:26:990.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:26:990.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692686990364 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:26:990.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:26:990.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:26:990.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:26:990.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692686990924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:26:991.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:26:991.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:26:991.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692686990965 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:26:991.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:27:038.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:038.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692687038136 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:038.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:038.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:038.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692687038321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42669 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:038.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:038.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:27:038.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:27:038.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:038.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692687038762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:038.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:564.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:564.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692687564312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:564.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:564.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:564.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692687564932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37989 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:564.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:565.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:27:565.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:27:565.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:565.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692687565384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:565.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:565.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:565.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692687565389 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:565.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:565.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:565.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692687565470 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:27:565.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:27:565.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:565.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692687565540 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:27:565.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:27:566.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:27:566.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:566.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692687566038 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:566.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:566.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:566.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692687566082 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:27:566.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:27:617.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:617.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692687617224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:617.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:617.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:617.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692687617806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46277 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:617.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:618.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:27:618.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:27:618.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:618.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692687618108 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:618.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:046.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:047.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692688046913 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:047.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:047.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:047.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692688047529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46769 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:047.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:047.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:28:047.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:28:047.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:048.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692688047983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:048.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:070.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:070.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692688070666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:070.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:071.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:071.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692688071091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44015 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:071.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:071.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:28:071.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:28:071.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:071.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692688071481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:071.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:554.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:554.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692688554296 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:554.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:554.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:554.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692688554884 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38613 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:554.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:555.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:28:555.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:28:555.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:555.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692688555183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:555.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:896.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:896.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692688896570 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:896.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:897.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:897.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692688897096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45587 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:897.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:897.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:28:897.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:28:897.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:897.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692688897727 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:897.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:897.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:897.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692688897734 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:897.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:897.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:897.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692688897814 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:28:898.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:28:898.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:898.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692688897919 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:28:898.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:28:898.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:898.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692688897938 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:28:898.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:28:898.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:898.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692688898501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:898.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:898.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:898.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:28:898.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692688898582 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:28:898.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:28:951.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:951.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692688951720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:951.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:952.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:952.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692688952207 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45379 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:952.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:28:952.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:28:952.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:28:952.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:28:952.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692688952669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:28:952.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:29:861.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:29:862.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692689861809 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:29:862.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:29:862.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:29:862.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692689861995 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43851 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:29:862.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:29:862.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:29:862.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:29:862.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:29:862.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692689862421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:29:862.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:29:956.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:29:956.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692689956171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:29:956.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:29:956.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:29:956.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692689956380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40711 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:29:956.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:29:956.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:29:956.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:29:956.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:29:956.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692689956781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:29:956.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:30:697.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:30:697.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692690697384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:30:697.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:30:697.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:30:697.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692690697824 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43855 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:30:697.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:30:698.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:30:698.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:30:698.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:30:698.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692690698111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:30:698.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:31:788.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:31:788.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692691788571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:31:788.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:31:788.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:31:788.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692691788762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46227 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:31:788.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:31:789.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:31:789.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:31:789.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:31:789.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692691789206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:31:789.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:31:833.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:31:833.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692691833517 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 5979 dequeues = 5979 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 6049 dequeues = 6049 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692662862656 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692691789215 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:11:31:833.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:31:833.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:31:833.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692691833644 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1192 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 107 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1358 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 326 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4604 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1316 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3003, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5979, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:31:834.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:31:841.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:31:841.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692691841639 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:11:31:841.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:11:31:841.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:31:841.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692691841682 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1714 in_bytes = 210968 out_tpkts = 1785 out_bytes = 99825 exception_packets = 60 exception_packets_allowed = 47 raw_vhost_stats= [ name = vhost0 in_pkts = 1714 in_bytes = 210968 out_pkts = 1785 out_bytes = 99825 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 6.78168e-15 total_out_bandwidth_utilization = 1.59757e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1251, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 2947, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1609 in_bytes = 92433 out_pkts = 1717 out_bytes = 211094 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:11:31:841.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:11:31:841.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:31:841.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692691841727 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:11:31:841.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:11:32:355.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:355.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692692355631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:32:355.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:32:356.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:356.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692692356148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42981 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:32:356.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:32:356.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:32:356.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:32:356.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:356.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692692356521 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:32:356.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:32:356.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:356.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692692356528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:32:356.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:32:356.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:356.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692692356589 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:32:356.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:32:356.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:356.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692692356682 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:32:356.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:32:356.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:356.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692692356741 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:32:356.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:32:357.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:357.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692692357290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:32:357.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:32:357.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:357.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692692357317 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:32:357.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:32:357.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:32:403.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:403.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692692403600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:32:403.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:32:403.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:403.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692692403773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43137 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:32:403.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:32:404.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:32:404.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:32:404.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:404.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692692404236 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:32:404.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:32:862.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:11:32:863.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:11:32:863.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:11:32:863.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692692862882 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:32:863.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:32:863.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:11:32:863.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692692863508 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1196 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 109 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1360 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 327 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4612 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1318 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3008, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5996, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:32:863.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:32:863.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:11:32:863.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:11:32:863.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:11:32:863.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:11:32:864.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:32:864.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Idle 2025-09-01 Mon 02:11:32:864.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:864.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692692864019 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1197 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 110 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1360 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 327 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4612 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1318 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3008, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5998, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:32:864.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:32:864.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:864.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692692864046 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:32:864.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:32:864.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:11:32:864.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:32:864.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692692864467 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1197 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 110 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1361 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 327 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4612 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1318 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3009, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5998, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:32:864.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:33:358.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:33:358.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692693358416 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:33:358.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:33:358.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:33:358.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692693358891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39883 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:33:358.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:33:359.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:33:359.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:33:359.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:33:359.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692693359530 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:33:359.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:168.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:169.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692694168959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:169.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:169.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:169.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692694169221 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35761 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:169.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:169.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:34:169.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:34:169.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:170.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692694169706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:170.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:170.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:170.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692694169711 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:170.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:170.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:170.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692694169787 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:34:170.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:34:170.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:170.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692694169841 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:34:170.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:34:170.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:170.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692694170407 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:170.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:170.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:170.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692694170456 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:34:170.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:34:170.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:34:219.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:219.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692694219612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:219.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:220.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:220.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692694220196 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46473 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:220.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:220.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:34:220.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:34:220.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:220.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692694220624 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:220.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:586.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:586.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692694586255 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:586.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:586.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:586.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692694586857 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34467 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:586.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:587.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:34:587.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:34:587.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:587.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692694587183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:587.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:587.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:587.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692694587192 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:587.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:587.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:587.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692694587287 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:34:587.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:34:587.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:587.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692694587337 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:34:587.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:34:587.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:587.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692694587833 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:587.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:587.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:587.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:34:587.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692694587878 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:34:587.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:34:638.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:638.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692694638058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:638.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:638.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:638.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692694638665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38713 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:638.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:34:639.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:34:639.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:34:639.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:34:639.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692694639194 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:34:639.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:219.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:219.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692695219779 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:219.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:220.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:220.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692695220258 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35941 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:220.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:220.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:35:220.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:35:220.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:220.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692695220837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:220.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:231.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:231.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692695231622 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:231.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:231.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:231.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692695231960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45001 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:231.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:232.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:35:232.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:35:232.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:232.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692695232465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:232.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:533.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:533.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692695533411 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:533.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:534.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:534.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692695533992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37995 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:534.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:534.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:35:534.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:35:534.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:534.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692695534389 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:534.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:534.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:534.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692695534396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:534.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:534.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:534.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692695534505 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:35:534.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:35:534.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:534.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692695534616 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:35:534.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:35:534.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:534.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692695534637 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:35:534.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:35:535.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:535.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:35:535.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692695535193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:535.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:535.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:535.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692695535231 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:35:535.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:35:583.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:583.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692695583325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:583.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:583.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:583.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692695583742 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38227 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:583.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:583.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:35:584.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:35:584.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:584.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692695584111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:584.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:620.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:620.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692695620359 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:620.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:620.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:620.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692695620809 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43361 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:620.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:35:621.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:35:621.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:35:621.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:35:621.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692695621325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:35:621.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:36:655.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:36:655.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692696655228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:36:655.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:36:656.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:36:656.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692696656140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44909 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:36:656.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:36:656.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:36:656.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:36:656.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:36:656.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692696656514 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:36:656.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:37:363.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:37:363.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692697363001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:37:363.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:37:363.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:37:363.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692697363631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34829 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:37:363.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:37:363.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:37:363.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:37:364.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:37:364.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692697363994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:37:364.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:37:727.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:37:727.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692697727501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:37:727.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:37:727.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:37:727.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692697727678 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38765 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:37:727.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:37:728.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:37:728.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:37:728.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:37:728.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692697728234 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:37:728.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:38:726.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:38:726.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692698726648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:38:726.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:38:727.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:38:727.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692698727085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40949 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:38:727.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:38:727.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:38:727.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:38:727.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:38:727.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692698727508 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:38:727.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:39:192.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:192.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692699192612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:39:192.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:39:193.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:193.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692699193091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38065 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:39:193.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:39:193.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:39:193.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:39:193.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:193.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692699193602 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:39:193.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:39:193.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:193.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692699193610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:39:193.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:39:193.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:193.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692699193679 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:39:193.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:39:194.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:194.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692699194032 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:39:194.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:39:194.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:194.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692699194050 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:39:194.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:39:194.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:194.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692699194647 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:39:194.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:39:194.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:194.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:39:194.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692699194689 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:39:194.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:39:243.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:244.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692699243932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:39:244.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:39:244.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:244.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692699244492 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42311 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:39:244.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:39:244.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:39:244.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:39:244.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:39:244.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692699244874 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:39:244.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:40:199.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:40:199.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:40:199.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692700199045 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:40:199.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:40:199.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:40:199.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:40:199.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692700199238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41365 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:40:199.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:40:199.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:40:199.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692700199746 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:40:199.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:508.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:508.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692701508445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:508.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:509.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:509.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692701509038 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35793 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:509.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:509.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:41:509.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:41:509.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:509.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692701509455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:509.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:509.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:509.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692701509462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:509.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:509.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:509.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692701509734 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:41:509.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:41:509.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:509.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692701509790 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:41:509.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:41:510.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:510.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692701510323 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:510.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:510.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:510.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:41:510.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692701510349 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:41:510.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:41:559.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:559.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692701559545 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:559.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:559.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:559.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692701559731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40187 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:559.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:560.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:41:560.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:41:560.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:560.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692701560147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:560.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:648.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:648.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692701648198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:648.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:648.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:648.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692701648762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36447 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:648.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:649.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:41:649.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:41:649.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:649.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692701649167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:649.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:649.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:649.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692701649176 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:649.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:649.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:649.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692701649273 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:41:649.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:41:649.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:649.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692701649336 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:41:649.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:41:650.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:650.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692701650091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:650.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:650.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:650.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692701650130 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:41:650.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:41:650.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:41:703.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:703.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692701703477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:703.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:703.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:703.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692701703868 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42887 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:703.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:704.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:41:704.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:41:704.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:704.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692701704411 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:704.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:323.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:324.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692702323916 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:324.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:324.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:324.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692702324130 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35103 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:324.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:324.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:42:324.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:42:324.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:324.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692702324578 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:324.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:568.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:568.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692702568247 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:568.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:568.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:568.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692702568447 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39547 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:568.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:568.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:42:568.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:42:568.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:568.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692702568804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:568.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:703.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:703.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692702703587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:703.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:704.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:704.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692702704082 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35411 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:704.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:704.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:42:704.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:42:704.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:704.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692702704574 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:704.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:795.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:795.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692702795665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:795.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:796.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:796.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692702796190 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42635 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:796.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:796.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:42:796.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:42:796.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:796.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692702796632 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:796.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:796.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:796.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692702796639 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:796.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:796.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:796.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692702796695 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:42:796.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:42:796.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:796.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692702796777 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:42:796.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:42:796.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:796.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692702796794 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:42:796.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:42:797.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:797.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692702797348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:797.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:797.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:797.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:42:797.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692702797388 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:42:797.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:42:849.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:849.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692702849605 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:849.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:849.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:849.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692702849811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33655 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:849.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:42:850.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:42:850.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:42:850.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:42:850.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692702850228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:42:850.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:43:768.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:43:768.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692703768421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:43:768.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:43:768.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:43:768.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692703768932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43319 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:43:768.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:43:769.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:43:769.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:43:769.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:43:769.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692703769697 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:43:769.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:44:558.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:44:558.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692704558697 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:44:558.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:44:559.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:44:559.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692704559619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33051 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:44:559.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:44:559.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:44:559.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:44:560.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:44:560.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692704559992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:44:560.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:44:675.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:44:675.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692704674957 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:44:675.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:44:676.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:44:676.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692704675994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42695 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:44:676.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:44:676.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:44:676.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:44:676.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:44:676.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692704676302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:44:676.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:45:677.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:45:677.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692705677860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:45:678.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:45:678.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:45:678.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692705678056 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46387 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:45:678.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:45:678.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:45:678.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:45:678.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:45:678.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692705678625 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:45:678.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:46:032.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:032.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692706032736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:032.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:46:033.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:033.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692706033299 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46437 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:033.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:46:033.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:46:033.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:46:033.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:033.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692706033806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:033.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:46:033.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:033.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692706033816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:033.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:46:033.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:033.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692706033897 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:46:033.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:46:034.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:034.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692706034032 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:46:034.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:46:034.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:034.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692706034058 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:46:034.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:46:034.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:034.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692706034581 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:034.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:46:034.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:034.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:46:034.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692706034606 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:46:034.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:46:085.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:085.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692706085785 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:085.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:46:086.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:086.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692706086349 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37675 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:086.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:46:086.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:46:086.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:46:086.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:086.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692706086660 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:086.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:47:004.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:47:004.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692707004832 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:47:004.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:47:005.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:47:005.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692707005501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40219 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:47:005.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:47:005.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:47:005.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:47:005.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:47:005.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692707005841 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:47:005.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:232.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:232.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692708232189 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:232.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:232.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:232.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692708232387 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33747 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:232.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:232.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:48:232.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:48:232.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:232.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692708232887 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:232.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:232.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:232.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692708232900 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:232.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:233.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:233.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692708232957 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:48:233.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:48:233.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:233.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692708233005 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:48:233.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:48:233.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:48:233.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:233.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692708233504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:233.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:233.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:233.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692708233587 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:48:233.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:48:281.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:281.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692708281734 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:281.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:281.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:281.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692708281903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44667 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:281.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:282.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:48:282.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:48:282.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:282.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692708282284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:282.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:924.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:924.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692708924463 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:924.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:925.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:925.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692708925042 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41933 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:925.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:925.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:48:925.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:48:925.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:925.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692708925377 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:925.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:925.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:925.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692708925384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:925.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:925.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:925.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692708925444 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:48:925.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:48:925.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:925.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692708925499 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:48:925.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:48:926.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:926.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692708926206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:926.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:48:926.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:926.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:926.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692708926250 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:48:926.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:48:968.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:968.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692708967976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:968.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:968.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:968.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692708968349 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45075 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:968.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:968.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:48:968.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:48:968.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:968.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692708968650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:968.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:979.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:979.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692708979310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:979.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:979.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:979.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692708979759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34841 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:979.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:980.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:48:980.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:48:980.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:980.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692708980265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:980.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:227.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:227.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692709227462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:49:227.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:227.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:227.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692709227642 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40503 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:49:227.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:227.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:49:227.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:49:228.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:228.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692709228040 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:49:228.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:907.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:907.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692709907405 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:49:907.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:908.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:908.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692709908090 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45779 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:49:908.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:908.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:49:908.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:49:908.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:908.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692709908542 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:49:908.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:962.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:962.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692709962736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:49:962.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:963.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:963.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692709963322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46771 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:49:963.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:963.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:49:963.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:49:963.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:963.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692709963741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:49:963.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:963.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:963.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692709963751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:49:963.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:963.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:963.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692709963813 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:49:963.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:49:964.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:964.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692709964057 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:49:964.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:49:964.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:964.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692709964480 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:49:964.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:49:965.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:965.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692709965004 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:49:965.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:49:965.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:49:965.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692709965046 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:49:965.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:49:965.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:50:015.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:50:015.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692710015220 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:50:015.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:50:015.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:50:015.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692710015763 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39197 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:50:015.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:50:016.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:50:016.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:50:016.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:50:016.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692710016540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:50:016.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:51:006.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:51:006.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692711006710 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:51:006.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:51:006.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:51:006.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692711006890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:51:006.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:51:007.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:51:007.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:51:007.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:51:007.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692711007340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:51:007.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:51:208.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:51:208.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692711208264 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:51:208.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:51:208.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:51:208.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692711208817 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46515 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:51:208.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:51:209.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:51:209.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:51:209.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:51:209.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692711209228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:51:209.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:51:780.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:51:780.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692711780682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:51:780.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:51:781.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:51:781.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692711781230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40015 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:51:781.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:51:781.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:51:781.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:51:781.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:51:781.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692711781665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:51:781.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:53:072.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:072.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692713072814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:53:072.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:53:073.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:073.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692713073310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46175 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:53:073.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:53:073.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:53:073.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:53:073.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:073.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692713073687 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:53:073.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:53:073.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:073.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692713073694 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:53:073.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:53:073.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:073.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692713073755 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:53:073.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:53:074.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:074.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692713074092 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:53:074.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:53:074.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:074.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692713074111 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:53:074.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:53:074.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:074.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692713074636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:53:074.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:53:074.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:074.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:53:074.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692713074676 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:53:074.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:53:122.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:122.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692713122781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:53:122.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:53:123.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:123.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692713123351 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39227 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:53:123.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:53:123.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:53:123.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:53:123.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:123.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692713123812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:53:123.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:53:149.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:149.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692713149474 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:53:149.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:53:149.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:149.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692713149685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43981 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:53:149.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:53:150.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:53:150.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:53:150.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:53:150.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692713150108 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:53:150.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:54:177.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:54:177.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692714176959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:54:177.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:54:177.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:54:177.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692714177549 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44179 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:54:177.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:54:177.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:54:177.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:54:178.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:54:178.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692714178010 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:54:178.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:349.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:349.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692715349380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:351.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:351.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:349.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:55:351.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692715349667 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42225 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:351.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:351.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:55:351.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:351.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692715351361 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:351.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:351.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:351.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692715351366 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:351.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:351.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:351.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692715351449 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:55:351.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:55:351.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:351.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692715351498 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:55:351.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:55:352.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:352.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692715352079 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:352.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:352.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:352.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:55:352.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692715352114 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:55:352.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:55:405.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:405.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692715405223 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:405.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:405.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:405.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692715405393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36037 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:405.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:405.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:55:405.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:55:405.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:405.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692715405732 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:405.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:993.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:993.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692715993823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:993.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:994.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:994.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692715994265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41371 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:994.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:994.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:55:994.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:55:994.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:994.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692715994770 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:994.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:994.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:994.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692715994776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:994.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:994.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:994.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692715994853 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:55:994.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:55:994.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:994.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692715994901 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:55:994.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:55:995.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:995.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692715995394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:55:995.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:55:995.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:995.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:55:995.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692715995419 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:55:995.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:56:041.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:041.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692716041601 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:041.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:56:041.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:041.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692716041794 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46273 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:041.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:56:042.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:56:042.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:56:042.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:042.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692716042231 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:042.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:56:104.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:104.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692716104119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:104.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:56:104.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:104.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692716104629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38997 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:104.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:56:104.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:56:105.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:56:105.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:105.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692716105153 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:105.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:56:323.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:323.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692716323889 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:324.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:56:324.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:324.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692716324522 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33485 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:324.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:56:324.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:56:324.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:56:324.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:324.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692716324818 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:324.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:56:960.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:960.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692716960429 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:960.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:56:960.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:960.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692716960648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32821 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:960.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:56:960.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:56:960.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:56:961.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:961.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692716961044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:56:961.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:57:182.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:182.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692717182314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:182.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:57:182.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:183.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692717182934 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37917 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:183.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:57:183.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:57:183.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:57:183.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:183.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692717183360 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:183.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:57:183.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:183.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692717183369 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:183.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:57:183.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:183.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692717183425 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:57:183.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:57:183.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:183.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692717183528 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:57:183.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:57:183.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:183.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692717183548 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:11:57:183.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:57:184.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:184.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692717184124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:184.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:57:184.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:184.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692717184179 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:57:184.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:57:184.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:11:57:230.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:230.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692717230305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:230.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:57:230.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:230.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692717230885 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36873 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:230.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:57:231.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:57:231.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:57:231.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:231.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692717231187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:231.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:58:275.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:58:275.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692718275335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:58:275.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:58:275.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:58:275.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692718275924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43529 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:58:276.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:58:276.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:58:276.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:58:276.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:58:276.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692718276232 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:58:276.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:58:341.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:58:341.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692718341013 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:58:341.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:58:341.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:58:341.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692718341456 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46193 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:58:341.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:58:341.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:58:341.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:58:341.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:58:341.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692718341916 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:58:341.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:58:779.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:58:779.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692718779192 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:58:779.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:58:779.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:58:779.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692718779782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35731 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:58:779.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:58:780.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:58:780.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:58:780.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:58:780.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692718780155 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:58:780.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:00:166.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:166.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692720166377 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:00:166.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:00:166.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:166.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692720166851 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37875 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:00:166.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:00:167.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:00:167.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:00:167.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:167.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692720167272 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:00:167.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:00:389.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:389.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692720389333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:00:390.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:00:391.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:00:391.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:391.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692720389650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42135 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:00:391.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:00:391.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:00:391.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:391.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692720391336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:00:391.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:00:391.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:391.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692720391352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:00:391.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:00:391.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:391.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692720391441 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:00:391.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:00:391.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:391.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692720391706 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:00:391.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:00:391.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:391.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692720391853 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:00:391.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:00:392.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:392.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692720392397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:00:392.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:00:392.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:392.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692720392446 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:00:392.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:00:392.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:00:439.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:439.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692720439637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:00:439.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:00:439.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:439.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692720439820 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39757 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:00:439.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:00:440.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:00:440.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:00:440.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:00:440.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692720440556 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:00:440.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:01:367.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:367.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692721367773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:01:367.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:01:367.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:367.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692721367953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35747 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:01:367.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:01:368.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:01:368.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:01:368.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:368.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692721368481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:01:368.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:01:834.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:834.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692721833834 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 6284 dequeues = 6284 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 6357 dequeues = 6357 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692692864430 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692721368488 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:12:01:834.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:01:834.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:834.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692721834014 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1254 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 112 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1426 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 343 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4839 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1383 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3155, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6284, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:01:834.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:01:841.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:841.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692721841818 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:12:01:841.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:12:01:841.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:841.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692721841863 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1379771104 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.19 five_min_avg = 0.1 fifteen_min_avg = 0.12 ] cpu_share = 1.09982 ] ] 2025-09-01 Mon 02:12:01:841.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:12:01:841.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:842.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692721841872 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.09982 used_sys_mem = 2854240503 one_min_cpuload = 0.19 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:12:01:842.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:12:01:842.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:842.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692721841905 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1789 in_bytes = 216294 out_tpkts = 1860 out_bytes = 103791 raw_vhost_stats= [ name = vhost0 in_pkts = 1789 in_bytes = 216294 out_pkts = 1860 out_bytes = 103791 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1681 in_bytes = 96273 out_pkts = 1792 out_bytes = 216420 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:12:01:842.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:12:01:842.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:842.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692721841936 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:12:01:842.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:12:01:942.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:942.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692721942085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:01:942.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:01:942.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:942.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692721942279 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34419 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:01:942.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:01:942.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:01:942.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:01:942.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:942.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692721942629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:01:942.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:01:942.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:942.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692721942635 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:01:942.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:01:942.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:942.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692721942711 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:01:942.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:01:942.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:942.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692721942781 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:01:942.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:01:943.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:943.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692721943296 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:01:943.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:01:943.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:943.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692721943339 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:01:943.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:01:943.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:01:990.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:990.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692721990529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:01:990.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:01:990.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:990.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692721990770 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44909 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:01:990.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:01:991.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:01:991.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:01:991.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:01:991.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692721991321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:01:991.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:02:596.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:596.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692722596316 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:02:596.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:02:596.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:596.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692722596915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32959 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:02:596.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:02:597.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:02:597.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:02:597.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:597.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692722597337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:02:597.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:02:597.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:597.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692722597342 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:02:597.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:02:597.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:597.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692722597415 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:02:597.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:02:597.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:597.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692722597462 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:02:597.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:02:598.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:598.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692722598316 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:02:598.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:02:598.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:598.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692722598375 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:02:598.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:02:598.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:02:651.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:651.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692722651494 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:02:651.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:02:651.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:651.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692722651973 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34795 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:02:652.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:02:652.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:02:652.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:02:652.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:652.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692722652394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:02:652.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:02:864.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:12:02:864.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:12:02:865.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:12:02:865.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692722864674 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:02:865.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:02:865.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:12:02:865.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:12:02:865.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:12:02:865.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692722864980 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1260 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 114 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1430 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 345 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4855 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1387 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3165, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6313, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:02:865.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:02:865.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:12:02:865.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:12:02:865.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:02:865.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Idle 2025-09-01 Mon 02:12:02:865.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:865.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692722865377 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1261 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 115 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1430 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 345 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4855 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1387 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3165, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6315, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:02:865.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:02:865.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:865.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692722865404 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:02:865.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:02:865.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:12:02:865.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:865.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692722865498 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1261 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 115 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1431 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 345 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4855 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1387 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6315, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:02:865.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:02:954.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:954.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692722954450 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:02:954.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:02:955.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:955.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692722955057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34313 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:02:955.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:02:955.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:02:955.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:02:955.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:02:955.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692722955541 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:02:955.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:03:330.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:03:330.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692723330662 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:03:330.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:03:331.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:03:331.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692723331294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44841 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:03:331.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:03:331.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:03:331.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:03:331.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:03:331.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692723331755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:03:331.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:03:642.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:03:642.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692723642539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:03:642.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:03:643.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:03:643.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692723643178 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34899 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:03:643.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:03:643.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:03:643.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:03:643.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:03:643.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692723643509 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:03:643.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:04:307.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:307.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692724307511 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:04:307.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:04:307.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:307.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692724307761 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35715 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:04:307.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:04:307.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:04:308.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:04:308.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:308.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692724308116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:04:308.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:04:308.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:308.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692724308122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:04:308.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:04:308.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:308.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692724308196 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:04:308.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:04:308.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:308.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692724308490 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:04:308.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:04:308.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:308.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692724308510 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:04:308.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:04:309.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:309.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692724309064 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:04:309.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:04:309.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:309.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692724309109 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:04:309.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:04:309.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:04:361.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:361.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692724361236 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:04:361.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:04:361.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:361.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692724361686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44605 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:04:361.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:04:362.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:04:362.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:04:362.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:362.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692724362409 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:04:362.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:04:809.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:809.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692724809760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:04:809.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:04:810.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:810.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692724810202 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35365 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:04:810.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:04:810.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:04:810.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:04:810.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:04:810.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692724810677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:04:810.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:05:379.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:05:379.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692725379521 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:05:379.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:05:379.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:05:379.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692725379713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39449 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:05:379.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:05:380.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:05:380.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:05:380.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:05:380.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692725380212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:05:380.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:05:713.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:05:713.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692725713618 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:05:713.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:05:714.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:05:714.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692725714197 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33869 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:05:714.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:05:714.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:05:714.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:05:714.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:05:714.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692725714821 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:05:714.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:07:414.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:414.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692727414284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:07:414.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:07:414.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:414.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692727414529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41601 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:07:414.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:07:414.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:07:414.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:07:415.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:415.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692727414993 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:07:415.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:07:471.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:472.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692727471944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:07:472.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:07:472.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:472.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692727472191 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46597 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:07:472.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:07:472.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:07:472.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:07:472.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:472.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692727472657 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:07:472.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:07:472.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:472.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692727472664 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:07:472.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:07:472.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:472.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692727472728 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:07:472.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:07:472.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:472.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692727472817 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:07:472.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:07:472.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:472.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692727472835 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:07:472.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:07:473.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:473.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692727473382 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:07:473.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:07:473.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:473.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692727473431 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:07:473.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:07:473.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:07:522.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:522.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692727522746 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:07:522.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:07:523.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:523.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692727523294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35015 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:07:523.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:07:523.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:07:523.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:07:523.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:07:523.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692727523629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:07:523.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:08:486.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:486.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692728486794 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:08:486.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:08:486.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:487.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692728486982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38859 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:08:487.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:08:487.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:08:487.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:08:487.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:487.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692728487421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:08:487.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:08:914.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:915.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692728914840 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:08:915.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:08:915.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:915.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692728915028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39819 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:08:915.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:08:915.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:08:915.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:08:915.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:916.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:08:916.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692728915495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:08:916.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:08:916.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:916.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692728915501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:08:916.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:08:916.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:916.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692728915559 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:08:916.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:08:916.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:916.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692728915612 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:08:916.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:08:916.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:916.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692728916156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:08:916.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:08:916.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:916.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692728916211 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:08:916.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:08:962.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:962.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692728962400 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:08:962.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:08:962.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:963.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692728962966 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34535 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:08:963.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:08:963.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:08:963.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:08:963.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:08:963.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692728963388 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:08:963.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:495.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:495.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692729494982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:09:495.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:495.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:495.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692729495564 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36601 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:09:495.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:495.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:09:495.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:09:495.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:496.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:09:496.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692729495805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:09:496.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:496.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:496.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692729495812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:09:496.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:496.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:496.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692729495870 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:09:496.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:09:496.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:496.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692729495915 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:09:496.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:09:496.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:496.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692729496481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:09:496.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:496.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:496.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692729496525 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:09:496.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:09:544.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:544.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692729544662 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:09:544.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:545.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:545.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692729545081 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:09:545.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:545.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:09:545.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:09:545.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:545.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692729545617 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:09:545.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:935.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:935.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692729935533 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:09:935.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:935.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:935.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692729935713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44283 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:09:935.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:936.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:09:936.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:09:936.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:09:936.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692729936122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:09:936.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:10:539.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:10:539.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692730539601 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:10:539.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:10:540.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:10:540.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692730540217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42749 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:10:540.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:10:540.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:10:540.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:10:540.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:10:540.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692730540510 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:10:540.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:10:580.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:10:580.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692730580723 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:10:580.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:10:580.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:10:580.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692730580880 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34009 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:10:580.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:10:581.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:10:581.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:10:581.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:10:581.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692730581347 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:10:581.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:11:123.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:123.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:11:124.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692731123132 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:11:124.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:11:124.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:124.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692731123630 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35927 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:11:124.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:11:125.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:11:125.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:125.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692731125101 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:11:125.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:11:125.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:125.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692731125107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:11:125.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:11:125.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:125.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692731125157 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:11:125.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:11:125.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:125.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692731125254 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:11:125.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:11:125.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:125.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692731125273 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:11:125.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:11:126.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:126.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692731125926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:11:126.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:11:126.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:126.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692731126039 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:11:126.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:11:126.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:11:171.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:171.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692731171444 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:11:171.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:11:171.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:171.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692731171889 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:11:171.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:11:172.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:11:172.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:11:172.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:11:172.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692731172369 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:11:172.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:060.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:060.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692732060280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:060.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:060.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:060.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692732060473 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39317 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:060.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:060.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:12:060.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:12:060.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:060.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692732060870 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:060.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:117.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:117.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692732117762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:117.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:118.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:118.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692732118281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41423 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:118.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:118.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:12:118.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:12:118.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:118.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692732118634 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:118.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:417.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:417.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692732417529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:417.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:417.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:417.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692732417720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39237 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:417.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:418.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:12:418.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:12:418.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:418.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692732418179 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:418.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:098.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:098.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692734098795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:098.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:099.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:099.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692734099295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37375 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:099.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:099.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:14:099.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:14:099.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:099.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692734099810 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:099.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:572.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:572.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692734572594 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:572.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:572.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:572.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692734572753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35553 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:572.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:572.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:14:572.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:14:573.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:573.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692734573041 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:573.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:573.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:573.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692734573047 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:573.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:573.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:573.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692734573133 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:14:573.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:14:573.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:573.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692734573227 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:14:573.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:14:573.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:573.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692734573246 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:14:573.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:14:573.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:573.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692734573837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:573.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:573.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:573.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:14:573.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692734573879 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:14:574.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:14:622.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:623.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692734622943 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:623.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:623.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:623.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692734623357 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42675 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:623.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:623.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:14:623.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:14:623.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:623.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692734623891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:623.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:15:596.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:596.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692735596026 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:15:596.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:15:596.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:596.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692735596488 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46663 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:15:596.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:15:596.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:15:596.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:15:597.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:597.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692735597008 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:15:597.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:15:805.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:805.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692735805058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:15:805.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:15:805.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:805.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692735805602 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33879 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:15:805.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:15:805.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:15:806.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:15:806.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:806.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692735806119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:15:806.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:15:806.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:806.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692735806126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:15:806.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:15:806.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:806.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692735806191 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:15:806.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:15:806.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:806.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692735806242 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:15:806.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:15:806.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:806.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692735806743 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:15:806.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:15:806.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:806.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692735806782 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:15:806.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:15:806.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:15:855.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:855.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692735855207 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:15:855.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:15:855.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:855.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692735855756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44329 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:15:855.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:15:855.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:15:855.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:15:856.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:15:856.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692735856045 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:15:856.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:16:126.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:127.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:16:127.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692736126294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:16:127.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:16:127.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:127.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692736126708 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35013 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:16:127.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:16:127.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:16:128.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:128.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692736128012 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:16:128.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:16:128.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:128.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692736128017 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:16:128.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:16:128.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:128.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692736128095 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:16:128.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:16:128.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:128.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692736128159 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:16:128.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:16:129.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:129.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692736129055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:16:129.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:16:129.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:16:129.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:129.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692736129097 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:16:129.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:16:179.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:179.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692736179281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:16:179.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:16:179.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:179.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692736179442 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40933 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:16:179.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:16:179.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:16:179.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:16:179.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:179.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692736179840 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:16:179.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:16:774.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:774.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692736774223 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:16:774.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:16:774.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:774.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692736774713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41363 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:16:774.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:16:774.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:16:774.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:16:775.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:16:775.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692736775006 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:16:775.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:116.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:116.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692737116017 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:116.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:116.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:116.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692737116228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35395 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:116.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:116.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:17:116.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:17:116.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:116.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692737116664 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:116.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:523.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:523.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692737523171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:523.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:523.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:523.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692737523343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40973 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:523.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:523.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:17:523.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:17:523.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:523.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692737523800 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:523.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:916.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:916.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692737916016 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:916.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:916.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:916.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692737916669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33133 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:916.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:917.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:17:917.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:17:917.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:917.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692737917142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:917.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:917.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:917.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692737917148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:917.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:917.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:917.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692737917214 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:17:917.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:17:917.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:917.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692737917302 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:17:917.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:17:917.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:917.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692737917319 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:17:917.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:17:918.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:918.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692737918089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:918.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:918.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:918.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:17:918.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692737918133 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:17:918.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:17:971.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:971.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692737971317 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:971.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:971.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:971.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692737971880 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39565 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:971.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:17:972.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:17:972.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:17:972.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:17:972.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692737972149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:17:972.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:18:881.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:18:881.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692738881217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:18:881.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:18:881.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:18:881.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692738881793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:18:881.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:18:882.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:18:882.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:18:882.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:18:882.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692738882379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:18:882.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:18:980.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:18:980.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692738980337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:18:980.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:18:980.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:18:980.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692738980884 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36291 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:18:980.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:18:981.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:18:981.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:18:981.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:18:981.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692738981348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:18:981.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:19:078.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:19:078.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692739078884 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:19:078.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:19:079.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:19:079.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692739079340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38191 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:19:079.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:19:079.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:19:079.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:19:079.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:19:079.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692739079856 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:19:079.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:20:835.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:20:835.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692740835477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:20:835.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:20:836.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:20:836.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692740836003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45463 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:20:836.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:20:836.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:20:836.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:20:836.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:20:836.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692740836345 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:20:836.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:20:941.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:40080 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:20:953.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:40094 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:21:772.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:772.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692741771963 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:21:772.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:21:772.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:772.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692741772146 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38207 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:21:772.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:21:772.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:21:772.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:21:772.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:773.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692741772588 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:21:773.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:21:773.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:773.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692741772593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:21:773.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:21:773.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:773.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692741772663 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:21:773.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:21:773.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:773.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692741772794 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:21:773.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:21:773.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:773.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692741772819 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:21:773.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:21:773.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:773.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692741773597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:21:773.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:21:773.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:773.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692741773666 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:21:773.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:21:773.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:21:823.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:823.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692741823831 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:21:823.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:21:824.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:824.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692741824290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41581 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:21:824.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:21:824.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:21:824.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:21:824.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:21:824.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692741824776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:21:824.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:716.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:716.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692742716030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:22:716.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:716.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:716.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692742716226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41255 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:22:716.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:716.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:22:716.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:22:716.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:716.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692742716605 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:22:716.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:716.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:716.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692742716611 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:22:716.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:716.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:716.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692742716757 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:22:716.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:22:716.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:716.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692742716869 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:22:716.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:22:717.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:717.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:22:717.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692742717452 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:22:717.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:717.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:717.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692742717496 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:22:718.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:22:770.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:770.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692742770628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:22:770.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:770.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:770.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692742770789 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41891 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:22:770.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:771.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:22:771.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:22:771.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:771.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692742771237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:22:771.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:859.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:860.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692742859955 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:22:860.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:860.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:860.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692742860139 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35889 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:22:860.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:860.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:22:860.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:22:860.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:22:860.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692742860554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:22:860.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:23:058.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:058.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692743058548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:23:058.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:23:059.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:059.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692743059128 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33771 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:23:059.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:23:059.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:23:059.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:23:059.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:059.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692743059470 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:23:059.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:23:059.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:059.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692743059479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:23:059.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:23:059.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:059.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692743059540 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:23:059.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:23:059.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:059.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692743059591 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:23:059.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:23:060.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:060.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692743060150 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:23:060.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:23:060.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:060.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692743060210 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:23:060.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:23:060.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:23:110.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:110.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692743110576 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:23:110.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:23:111.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:111.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692743111207 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32769 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:23:111.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:23:111.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:23:111.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:23:111.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:111.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692743111633 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:23:111.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:23:779.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:779.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692743779465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:23:779.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:23:779.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:779.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692743779653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35661 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:23:779.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:23:780.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:23:780.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:23:780.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:23:780.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692743780128 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:23:780.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:137.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:137.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692744137742 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:137.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:138.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:138.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692744138191 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45599 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:138.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:138.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:24:138.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:24:138.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:138.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692744138616 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:138.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:724.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:724.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692744724554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:724.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:725.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:725.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692744725153 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34635 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:725.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:725.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:24:725.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:24:725.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:725.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692744725463 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:725.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:725.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:725.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692744725470 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:725.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:725.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:725.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692744725647 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:24:725.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:24:726.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:726.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692744726114 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:24:726.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:24:726.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:726.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692744726133 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:24:726.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:24:726.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:726.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692744726739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:726.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:726.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:726.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692744726830 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:24:726.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:24:726.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:24:779.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:779.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692744778929 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:779.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:779.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:779.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692744779132 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36573 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:779.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:779.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:24:779.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:24:779.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:779.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692744779569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:779.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:858.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:858.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692744858713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:858.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:859.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:859.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692744859209 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33615 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:859.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:24:859.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:24:859.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:24:859.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:859.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692744859675 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:24:859.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:25:697.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:697.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692745697701 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:25:697.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:25:699.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:25:699.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:25:699.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:699.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692745699052 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36153 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:25:699.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:25:699.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:699.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692745699120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:25:699.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:25:724.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:724.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692745724277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:25:724.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:25:724.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:724.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692745724426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39267 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:25:724.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:25:724.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:25:724.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:25:724.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:724.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692745724834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:25:724.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:25:974.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:974.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692745974755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:25:974.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:25:975.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:975.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692745975269 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35473 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:25:975.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:25:975.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:25:975.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:25:975.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:975.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692745975726 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:25:975.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:27:517.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:27:517.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692747517319 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:27:518.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:27:519.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:27:519.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:27:519.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692747517896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39735 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:27:519.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:27:519.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:27:519.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:27:519.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692747519129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:27:519.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:28:711.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:711.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692748711844 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:28:711.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:28:712.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:712.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692748712488 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41771 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:28:712.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:28:712.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:28:712.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:28:712.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:713.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692748712915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:28:713.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:28:713.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:713.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692748712921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:28:713.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:28:713.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:713.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692748712993 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:28:713.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:28:713.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:713.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692748713271 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:28:713.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:28:713.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:713.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692748713292 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:28:713.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:28:714.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:714.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692748714305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:28:714.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:28:714.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:714.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692748714353 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:28:714.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:28:714.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:28:765.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:765.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692748765561 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:28:765.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:28:765.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:765.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692748765969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37053 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:28:766.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:28:766.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:28:766.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:28:766.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:28:766.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692748766424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:28:766.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:505.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:505.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749505000 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:505.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:505.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:505.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692749505677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39233 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:505.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:506.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:29:506.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:29:506.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:506.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692749506103 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:506.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:506.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:506.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692749506111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:506.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:506.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:506.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692749506226 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:29:506.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:29:506.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:506.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692749506274 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:29:506.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:29:506.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:506.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:29:506.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749506778 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:506.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:506.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:506.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749506819 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:29:507.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:29:557.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:558.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692749557965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:558.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:558.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:558.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692749558613 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37459 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:558.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:558.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:29:558.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:29:559.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:559.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749559040 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:559.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:575.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:575.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692749575815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:575.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:576.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:576.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692749576260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41509 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:576.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:576.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:29:576.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:29:576.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:576.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749576684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:576.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:576.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:576.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749576701 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:576.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:576.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:576.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749576763 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:29:576.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:29:576.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:576.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749576813 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:29:576.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:29:577.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:577.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749577325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:577.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:577.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:577.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749577368 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:29:577.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:29:577.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:29:624.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:624.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692749624498 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:624.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:625.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:625.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692749625048 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38453 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:625.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:625.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:29:625.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:29:625.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:625.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692749625482 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:625.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:702.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:702.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749702619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:702.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:703.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:703.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692749703267 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36177 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:703.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:703.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:29:703.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:29:703.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:703.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692749703622 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:703.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:30:486.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:30:486.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692750486192 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:30:486.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:30:486.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:30:486.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692750486393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34961 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:30:486.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:30:486.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:30:486.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:30:486.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:30:486.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692750486799 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:30:486.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:30:660.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:30:660.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692750660635 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:30:660.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:30:661.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:30:661.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692750661310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39345 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:30:661.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:30:661.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:30:662.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:30:662.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:30:662.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692750662316 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:30:662.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:299.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:299.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692751299311 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:31:299.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:299.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:299.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692751299874 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38295 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:31:299.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:300.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:31:300.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:31:300.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:300.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692751300445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:31:300.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:300.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:300.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692751300452 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:31:300.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:300.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:300.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692751300532 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:31:300.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:31:300.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:300.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692751300628 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:31:300.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:31:300.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:300.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692751300646 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:31:300.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:31:301.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:301.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692751301391 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:31:301.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:301.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:301.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692751301445 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:31:301.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:31:301.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:31:349.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:349.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692751349587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:31:349.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:350.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:350.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692751350187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33835 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:31:350.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:350.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:31:350.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:31:350.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:350.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692751350706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:31:350.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:557.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:557.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692751557734 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:31:557.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:558.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:558.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692751558323 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36437 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:31:558.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:558.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:31:558.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:31:558.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:559.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692751558614 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:31:559.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:31:834.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:834.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692751834183 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 6619 dequeues = 6619 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 6695 dequeues = 6695 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692722865461 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692751558987 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:12:31:834.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:31:834.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:834.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692751834323 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1322 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 117 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1502 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 362 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5096 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1456 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3323, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6619, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:31:834.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:31:842.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:842.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692751842187 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:12:31:842.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:12:31:842.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:842.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692751842240 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1883 in_bytes = 233241 out_tpkts = 1959 out_bytes = 109661 exception_packets = 63 exception_packets_allowed = 50 raw_vhost_stats= [ name = vhost0 in_pkts = 1883 in_bytes = 233241 out_pkts = 1959 out_bytes = 109661 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] phy_if_5min_usage= [ [ [ name = ens6 in_bandwidth_usage = 1258 out_bandwidth_usage = 2951 ], ] ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 6.92263e-15 total_out_bandwidth_utilization = 1.6095e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1277, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 2969, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1777 in_bytes = 102017 out_pkts = 1886 out_bytes = 233367 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:12:31:842.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:12:31:842.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:31:843.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692751842936 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:12:31:843.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:12:32:358.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:358.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692752358825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:32:358.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:32:359.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:359.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692752359327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40963 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:32:359.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:32:359.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:32:359.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:32:359.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:359.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692752359848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:32:359.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:32:430.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:431.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692752430943 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:32:431.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:32:431.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:431.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692752431456 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42931 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:32:431.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:32:431.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:32:431.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:32:431.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:431.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692752431855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:32:431.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:32:480.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:480.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692752480499 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:32:480.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:32:480.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:480.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692752480674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39415 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:32:480.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:32:481.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:32:481.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:32:481.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:481.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692752481177 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:32:481.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:32:865.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:12:32:867.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:12:32:867.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:12:32:867.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692752867356 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:32:867.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:32:867.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:12:32:867.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692752867644 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1322 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 119 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1505 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 362 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5105 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1459 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3329, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6633, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:32:867.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:32:867.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:12:32:867.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:12:32:868.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:12:32:868.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:12:32:868.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:32:868.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Idle 2025-09-01 Mon 02:12:32:868.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:868.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692752868091 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1323 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 120 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1505 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 362 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5105 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1459 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3329, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6635, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:32:868.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:32:868.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:868.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692752868141 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:32:868.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:32:868.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:12:32:868.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:32:868.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692752868311 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1323 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 120 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1506 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 362 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5105 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1459 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3330, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6635, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:32:868.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:34:466.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:34:466.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692754466034 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:34:466.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:34:466.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:34:466.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692754466404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42569 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:34:466.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:34:466.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:34:466.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:34:466.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:34:466.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692754466948 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:34:466.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:35:518.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:518.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692755518753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:35:518.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:35:519.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:519.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692755519226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45701 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:35:519.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:35:519.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:35:519.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:35:519.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:519.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692755519587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:35:519.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:35:519.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:519.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692755519599 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:35:519.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:35:519.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:519.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692755519678 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:35:519.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:35:519.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:519.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692755519780 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:35:519.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:35:519.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:519.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692755519799 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:35:519.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:35:520.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:520.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692755520336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:35:520.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:35:520.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:520.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692755520406 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:35:520.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:35:520.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:35:570.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:570.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692755570529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:35:570.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:35:570.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:570.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692755570721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38509 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:35:570.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:35:571.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:35:571.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:35:571.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:35:571.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692755571139 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:35:571.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:153.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:153.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692756153297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:153.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:153.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:153.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692756153783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34331 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:153.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:154.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:36:154.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:36:154.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:154.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692756154321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:154.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:154.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:154.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692756154326 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:154.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:154.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:154.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692756154386 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:36:154.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:36:154.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:154.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692756154435 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:36:154.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:36:154.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:154.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692756154927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:154.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:154.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:155.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692756154970 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:36:155.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:36:155.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:36:202.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:202.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692756202272 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:202.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:202.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:202.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692756202661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45965 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:202.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:202.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:36:203.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:36:203.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:203.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692756203087 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:203.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:248.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:248.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692756248030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:248.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:248.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:248.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692756248215 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44819 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:248.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:248.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:36:248.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:36:248.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:248.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692756248569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:248.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:248.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:248.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692756248583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:248.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:248.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:248.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692756248643 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:36:248.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:36:248.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:248.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692756248691 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:36:248.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:36:249.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:249.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692756249240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:249.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:249.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:249.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692756249282 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:36:249.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:36:249.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:36:301.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:301.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692756301805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:301.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:302.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:302.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692756302362 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46841 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:302.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:302.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:36:302.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:36:302.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:302.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692756302832 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:302.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:534.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:534.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692756534308 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:534.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:534.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:534.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692756534490 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42557 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:534.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:36:534.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:36:534.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:36:534.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:36:534.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692756534950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:36:534.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:37:265.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:37:265.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692757265236 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:37:265.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:37:265.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:37:265.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692757265439 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44295 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:37:265.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:37:265.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:37:265.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:37:265.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:37:265.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692757265872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:37:265.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:37:364.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:37:365.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692757364957 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:37:365.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:37:365.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:37:365.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692757365126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43321 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:37:365.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:37:365.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:37:365.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:37:365.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:37:365.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692757365582 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:37:365.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:38:569.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:569.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692758569113 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:38:569.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:38:569.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:569.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692758569315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40213 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:38:569.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:38:569.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:38:569.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:38:569.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:569.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692758569742 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:38:569.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:38:715.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:715.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692758715082 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:38:715.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:38:715.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:715.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692758715625 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33161 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:38:715.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:38:715.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:38:716.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:38:716.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:716.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692758716115 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:38:716.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:38:716.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:716.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692758716121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:38:716.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:38:716.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:716.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692758716214 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:38:716.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:38:716.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:716.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692758716301 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:38:716.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:38:716.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:716.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692758716319 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:38:716.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:38:716.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:716.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692758716844 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:38:716.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:38:716.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:716.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692758716880 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:38:716.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:38:716.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:38:763.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:763.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692758763019 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:38:763.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:38:763.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:763.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692758763526 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33111 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:38:763.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:38:763.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:38:763.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:38:763.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:38:763.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692758763937 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:38:763.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:39:408.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:39:408.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692759408072 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:39:408.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:39:408.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:39:408.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692759408654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35607 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:39:408.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:39:408.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:39:409.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:39:409.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:39:409.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692759409090 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:39:409.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:39:507.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:39:507.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692759507740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:39:507.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:39:507.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:39:507.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692759507931 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41285 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:39:507.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:39:508.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:39:508.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:39:508.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:39:508.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692759508350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:39:508.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:39:808.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:39:808.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692759808084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:39:808.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:39:808.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:39:808.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692759808878 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34973 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:39:808.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:39:809.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:39:809.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:39:809.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:39:809.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692759809229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:39:809.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:41:609.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:41:609.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692761609349 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:41:609.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:41:609.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:41:610.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692761609946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37931 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:41:610.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:41:610.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:41:610.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:41:610.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:41:610.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692761610289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:41:610.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:781.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:782.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692762781932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:782.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:782.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:782.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692762782107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43873 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:782.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:782.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:42:782.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:42:782.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:782.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692762782607 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:782.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:782.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:782.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692762782613 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:782.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:782.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:782.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692762782686 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:42:782.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:42:782.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:782.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692762782767 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:42:783.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:42:783.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:783.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692762782784 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:42:783.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:42:783.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:783.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692762783314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:783.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:783.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:783.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692762783353 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:42:783.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:42:783.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:42:833.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:833.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692762833504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:833.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:833.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:833.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692762833702 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39735 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:833.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:834.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:42:834.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:42:834.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:834.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692762834124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:834.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:153.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:154.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692763153253 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:153.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:43:154.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:154.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:43:155.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:155.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692763153432 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42909 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:155.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:155.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:155.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692763155066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:155.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:155.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:155.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692763155111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:155.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:155.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:155.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692763155165 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:43:155.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:43:155.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:155.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692763155238 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:43:155.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:43:155.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:155.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692763155888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:155.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:155.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:155.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:43:155.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692763155952 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:43:156.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:43:207.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:207.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692763207059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:207.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:207.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:207.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692763207640 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:207.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:207.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:43:207.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:43:207.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:207.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692763207909 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:207.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:720.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:720.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692763720559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:720.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:720.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:720.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692763720795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45945 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:720.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:721.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:43:721.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:43:721.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:721.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692763721263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:721.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:721.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:721.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692763721269 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:721.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:721.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:721.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692763721344 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:43:721.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:43:721.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:721.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692763721393 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:43:721.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:43:721.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:721.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692763721949 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:722.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:722.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:722.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692763721989 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:43:722.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:43:722.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:43:770.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:770.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692763770277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:770.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:770.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:770.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692763770463 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41577 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:770.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:770.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:43:770.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:43:770.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:770.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692763770861 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:770.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:775.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:775.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692763775081 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:775.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:775.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:775.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692763775198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45729 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:775.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:43:775.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:43:775.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:43:775.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:43:775.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692763775530 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:43:775.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:44:216.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:44:216.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692764216053 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:44:216.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:44:216.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:44:216.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692764216271 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43075 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:44:216.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:44:216.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:44:216.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:44:216.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:44:216.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692764216686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:44:216.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:44:801.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:44:802.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692764801676 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:44:802.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:44:802.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:44:802.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692764801850 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35859 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:44:802.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:44:802.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:44:802.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:44:802.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:44:802.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692764802583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:44:802.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:45:462.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:462.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692765462375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:45:462.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:45:462.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:462.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692765462816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43227 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:45:462.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:45:463.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:45:463.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:45:463.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:463.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692765463260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:45:463.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:45:463.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:463.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692765463267 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:45:463.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:45:463.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:463.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692765463323 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:45:463.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:45:463.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:463.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692765463419 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:45:463.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:45:463.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:463.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692765463437 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:45:463.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:45:464.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:464.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692765464036 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:45:464.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:45:464.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:464.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692765464078 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:45:464.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:45:464.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:45:513.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:513.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692765513297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:45:513.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:45:513.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:513.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692765513570 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34795 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:45:513.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:45:513.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:45:513.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:45:513.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:514.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692765513984 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:45:514.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:45:679.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:679.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692765679020 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:45:679.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:45:679.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:679.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692765679236 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35511 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:45:679.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:45:679.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:45:679.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:45:679.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:45:679.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692765679661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:45:679.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:46:286.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:46:286.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692766286830 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:46:286.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:46:287.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:46:287.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692766287281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34725 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:46:287.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:46:287.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:46:287.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:46:287.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:46:287.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692766287583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:46:287.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:46:450.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:46:450.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692766450209 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:46:450.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:46:450.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:46:450.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692766450815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46413 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:46:450.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:46:451.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:46:451.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:46:451.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:46:451.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692766451335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:46:451.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:46:602.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:46:602.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692766602759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:46:602.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:46:603.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:46:603.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692766603270 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33647 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:46:603.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:46:603.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:46:603.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:46:603.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:46:603.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692766603792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:46:603.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:48:341.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:48:343.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692768341489 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:48:342.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:48:343.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:48:343.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:48:343.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:48:343.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692768342004 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44971 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:48:344.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:48:344.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:48:344.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692768343876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:48:344.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:49:315.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:315.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692769315819 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:49:315.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:49:316.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:316.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692769316338 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35769 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:49:316.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:49:316.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:49:316.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:49:316.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:316.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692769316848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:49:316.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:49:316.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:316.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692769316854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:49:316.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:49:316.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:316.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692769316908 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:49:316.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:49:316.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:317.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692769316990 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:49:317.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:49:317.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:317.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692769317008 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:49:317.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:49:317.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:317.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692769317585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:49:317.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:49:317.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:317.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692769317625 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:49:317.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:49:317.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:49:362.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:362.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692769362801 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:49:362.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:49:362.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:362.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692769362964 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39113 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:49:363.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:49:363.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:49:363.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:49:363.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:49:363.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692769363276 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:49:363.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:319.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:319.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692770319730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:319.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:320.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:320.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692770320305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38843 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:320.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:320.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:50:320.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:50:320.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:320.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692770320844 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:320.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:320.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:320.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692770320848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:320.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:320.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:320.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692770320902 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:50:320.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:50:320.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:320.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692770320957 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:50:320.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:50:321.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:321.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692770321498 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:321.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:321.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:321.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692770321619 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:50:321.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:50:321.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:50:370.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:370.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692770369977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:370.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:370.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:370.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692770370397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34497 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:370.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:370.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:50:371.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:50:371.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:371.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692770371081 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:371.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:416.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:416.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692770416391 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:416.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:416.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:416.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692770416560 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:416.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:416.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:50:416.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:50:416.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:416.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692770416878 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:416.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:635.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:636.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692770635931 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:636.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:636.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:636.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692770636528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34183 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:636.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:636.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:50:636.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:50:636.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:637.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692770636918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:637.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:637.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:637.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692770636925 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:637.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:637.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:637.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692770636984 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:50:637.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:50:637.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:637.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692770637038 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:50:637.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:50:637.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:637.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692770637749 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:637.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:637.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:637.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692770637789 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:50:637.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:50:637.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:50:686.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:687.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692770686961 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:687.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:687.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:687.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692770687587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42639 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:687.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:50:687.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:50:687.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:50:687.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:50:687.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692770687894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:50:687.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:51:433.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:51:433.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692771433246 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:51:433.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:51:433.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:51:433.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692771433833 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33027 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:51:433.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:51:434.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:51:434.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:51:434.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:51:434.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692771434336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:51:434.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:51:687.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:51:687.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692771687070 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:51:687.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:51:687.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:51:687.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:51:687.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:51:687.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692771687256 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39179 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:51:687.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:51:687.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:51:687.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692771687591 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:51:687.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:52:196.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:196.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692772196001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:52:196.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:52:196.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:196.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692772196213 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44347 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:52:196.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:52:196.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:52:196.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:52:196.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:196.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692772196680 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:52:196.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:52:196.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:196.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692772196695 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:52:196.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:52:196.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:196.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692772196754 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:52:196.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:52:196.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:196.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692772196862 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:52:196.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:52:196.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:196.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692772196880 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:52:196.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:52:197.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:197.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692772197445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:52:197.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:52:197.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:197.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692772197490 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:52:197.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:52:197.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:52:248.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:248.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692772248662 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:52:248.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:52:248.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:248.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692772248855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40837 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:52:248.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:52:249.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:52:249.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:52:249.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:249.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692772249277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:52:249.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:52:361.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:361.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692772361055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:52:361.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:52:361.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:361.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692772361637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34299 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:52:361.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:52:361.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:52:361.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:52:361.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:52:362.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692772361960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:52:362.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:53:203.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:53:203.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692773203433 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:53:203.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:53:203.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:53:203.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692773203814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40993 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:53:203.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:53:204.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:53:204.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:53:204.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:53:204.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692773204292 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:53:204.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:53:396.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:53:398.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692773396503 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:53:397.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:53:398.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:53:398.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:53:398.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692773396713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35757 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:53:398.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:53:398.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:53:398.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:53:398.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692773398578 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:53:398.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:53:685.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:53:685.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692773685714 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:53:685.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:53:686.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:53:686.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692773686144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40159 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:53:686.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:53:686.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:53:686.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:53:686.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:53:686.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692773686476 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:53:686.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:55:202.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:55:202.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692775202428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:55:202.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:55:202.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:55:202.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692775202906 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33613 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:55:202.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:55:203.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:55:203.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:55:203.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:55:203.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692775203361 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:55:203.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:56:070.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:070.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692776070152 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:56:070.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:56:070.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:070.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692776070705 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39991 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:56:070.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:56:070.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:56:071.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:56:071.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:071.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692776071156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:56:071.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:56:071.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:071.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692776071162 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:56:071.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:56:071.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:071.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692776071232 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:56:071.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:56:071.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:071.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692776071362 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:56:071.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:56:071.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:071.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692776071388 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:56:071.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:56:072.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:072.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692776072093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:56:072.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:56:072.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:072.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:56:072.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692776072155 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:56:072.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:56:122.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:122.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692776122385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:56:122.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:56:122.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:122.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692776122961 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46879 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:56:122.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:56:123.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:56:123.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:56:123.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:123.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692776123756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:56:123.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:158.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:159.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692777158907 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:159.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:159.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:159.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692777159085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33525 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:159.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:159.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:57:159.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:57:159.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:159.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692777159521 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:159.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:394.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:394.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692777394589 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:394.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:395.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:395.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692777395030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35431 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:395.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:395.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:57:395.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:57:395.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:395.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692777395397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:395.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:395.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:395.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692777395402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:395.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:395.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:395.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692777395455 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:57:395.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:57:395.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:395.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692777395500 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:57:395.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:57:396.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:396.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692777396008 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:396.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:396.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:396.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:57:396.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692777396041 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:57:396.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:57:430.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:430.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692777430738 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:430.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:431.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:431.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692777431153 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45199 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:431.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:431.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:57:431.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:57:431.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:431.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692777431570 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:431.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:431.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:431.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692777431576 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:431.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:431.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:431.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692777431662 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:57:431.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:57:431.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:431.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692777431719 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:57:431.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:57:432.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:432.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692777432261 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:432.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:432.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:432.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:57:432.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692777432302 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:57:432.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:57:445.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:445.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692777445169 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:445.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:445.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:445.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692777445327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42597 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:445.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:445.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:57:445.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:57:445.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:445.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692777445667 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:445.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:484.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:484.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692777484414 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:484.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:484.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:484.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692777484772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38985 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:484.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:57:484.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:57:485.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:57:485.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:57:485.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692777485038 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:57:485.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:381.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:382.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692778381834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:382.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:382.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:382.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692778382015 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37933 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:382.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:382.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:58:382.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:58:382.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:382.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692778382402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:382.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:538.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:538.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:58:539.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692778538202 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:539.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:539.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:539.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692778538370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34935 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:539.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:539.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:58:539.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:539.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692778539941 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:539.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:983.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:983.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692778983473 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:983.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:983.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:983.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692778983672 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34005 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:983.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:983.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:58:984.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:58:984.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:984.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692778984106 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:984.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:984.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:984.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692778984113 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:984.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:984.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:984.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692778984197 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:58:984.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:58:984.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:984.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692778984284 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:58:984.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:58:984.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:984.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692778984303 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:12:58:984.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:58:984.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:984.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692778984873 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:984.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:984.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:984.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692778984917 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:58:984.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:58:985.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:12:59:033.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:59:033.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692779033205 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:59:033.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:59:033.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:59:033.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692779033408 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35971 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:59:033.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:59:033.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:59:033.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:59:033.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:59:033.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692779033776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:59:033.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:59:103.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:59:103.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692779103758 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:59:103.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:59:104.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:59:104.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692779104558 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32995 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:59:104.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:59:104.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:59:104.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:59:105.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:59:105.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692779105055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:59:105.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:00:077.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:00:078.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692780077936 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:00:078.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:00:078.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:00:078.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692780078472 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42659 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:00:078.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:00:078.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:00:078.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:00:078.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:00:078.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692780078863 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:00:078.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:00:308.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:00:308.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692780308538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:00:308.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:00:308.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:00:308.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692780308971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37049 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:00:308.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:00:309.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:00:309.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:00:309.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:00:309.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692780309457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:00:309.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:00:502.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:00:502.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692780502075 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:00:502.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:00:502.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:00:502.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:00:505.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:00:505.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692780502536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46053 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:00:505.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:00:505.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:00:505.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692780502857 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:00:505.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:01:834.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:01:834.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692781834478 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 6918 dequeues = 6918 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 6997 dequeues = 6997 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692752868251 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692780505601 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:13:01:834.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:01:834.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:01:834.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692781834616 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1380 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 122 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1570 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 378 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5329 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1523 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3474, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6918, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:01:835.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:01:842.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:01:842.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692781842408 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:13:01:842.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:13:01:842.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:01:842.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692781842456 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1375572704 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.195 five_min_avg = 0.115 fifteen_min_avg = 0.125 ] cpu_share = 1.10833 ] ] 2025-09-01 Mon 02:13:01:842.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:13:01:842.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:01:842.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692781842462 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.10833 used_sys_mem = 2854240503 one_min_cpuload = 0.195 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:13:01:842.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:13:01:842.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:01:842.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692781842490 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1958 in_bytes = 238567 out_tpkts = 2034 out_bytes = 113627 exception_packets = 64 exception_packets_allowed = 51 raw_vhost_stats= [ name = vhost0 in_pkts = 1958 in_bytes = 238567 out_pkts = 2034 out_bytes = 113627 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1849 in_bytes = 105857 out_pkts = 1961 out_bytes = 238693 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:13:01:842.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:13:01:842.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:01:842.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692781842514 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:13:01:842.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:13:01:969.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:01:969.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692781969038 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:01:969.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:01:969.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:01:969.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692781969735 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41153 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:01:969.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:01:969.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:01:970.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:01:970.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:01:970.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692781970125 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:01:970.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:02:868.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:13:02:868.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:13:02:868.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:13:02:868.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692782868487 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:02:868.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:02:868.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:13:02:868.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:13:02:868.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:13:02:869.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692782868778 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1380 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 124 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1571 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 378 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5332 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1524 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3476, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6928, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:02:869.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:02:869.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:13:02:869.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:13:02:869.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:02:869.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Idle 2025-09-01 Mon 02:13:02:869.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:02:869.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692782869095 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1381 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 125 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1571 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 378 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5332 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1524 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3476, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6930, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:02:869.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:02:869.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:02:869.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692782869125 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:02:869.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:02:869.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:13:02:869.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:02:869.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692782869241 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1381 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 125 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1572 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 378 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5332 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1524 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3477, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6930, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:02:869.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:03:065.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:065.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692783065218 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:03:065.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:03:065.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:065.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692783065878 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:03:065.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:03:066.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:03:066.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:03:066.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:066.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692783066930 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:03:066.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:03:066.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:067.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692783066942 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:03:067.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:03:067.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:067.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692783067006 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:03:067.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:03:067.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:067.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692783067109 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:03:067.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:03:067.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:067.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692783067136 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:03:067.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:03:067.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:067.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692783067867 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:03:067.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:03:067.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:067.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692783067922 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:03:067.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:03:067.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:03:117.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:117.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692783117077 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:03:117.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:03:117.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:117.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692783117689 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41839 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:03:117.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:03:117.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:03:118.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:03:118.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:03:118.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692783118062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:03:118.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:045.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:046.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:04:046.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692784045265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:046.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:047.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:047.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692784045854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33747 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:047.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:047.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:04:047.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:047.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692784047110 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:047.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:377.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:377.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692784377709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:377.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:378.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:378.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692784378216 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32901 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:378.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:378.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:04:378.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:04:378.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:378.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692784378803 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:378.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:378.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:378.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692784378811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:378.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:378.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:378.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692784378903 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:04:378.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:04:378.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:379.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692784378985 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:04:379.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:04:379.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:379.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692784379624 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:379.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:379.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:379.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692784379668 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:04:379.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:04:379.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:04:390.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:390.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692784390931 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:390.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:391.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:391.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692784391394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45417 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:391.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:391.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:04:391.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:04:391.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:391.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692784391720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:391.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:391.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:391.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692784391728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:391.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:391.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:391.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692784391786 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:04:391.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:04:391.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:391.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692784391847 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:04:391.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:04:392.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:392.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692784392428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:392.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:392.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:392.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692784392458 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:04:392.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:04:392.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:04:429.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:429.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692784429888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:429.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:430.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:430.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692784430385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41501 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:430.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:430.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:04:430.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:04:430.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:430.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692784430943 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:431.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:443.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:443.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692784443661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:443.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:443.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:443.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692784443976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44487 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:444.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:04:444.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:04:444.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:04:444.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:04:444.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692784444391 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:04:444.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:05:362.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:05:362.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692785362519 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:05:362.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:05:363.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:05:363.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692785363423 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35259 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:05:363.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:05:363.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:05:363.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:05:363.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:05:363.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692785363793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:05:363.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:05:502.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:05:502.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692785502075 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:05:502.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:05:502.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:05:502.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692785502532 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36785 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:05:502.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:05:502.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:05:503.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:05:503.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:05:503.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692785503115 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:05:503.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:05:991.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:05:991.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692785991218 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:05:991.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:05:991.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:05:991.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692785991396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36347 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:05:991.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:05:991.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:05:991.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:05:991.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:05:991.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692785991815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:05:991.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:06:146.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:06:146.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692786146314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:06:146.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:06:146.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle