2025-09-01 Mon 02:04:35:511.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:35:511.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:511.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275511513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:511.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:511.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:511.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275511518 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:511.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:511.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:511.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275511571 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:35:511.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:35:511.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:511.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275511623 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:35:511.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:35:512.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:512.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692275512112 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:512.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:512.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:512.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692275512159 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:35:512.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:35:512.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:35:559.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:559.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692275559457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:559.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:560.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:560.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692275559990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44593 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:560.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:560.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:35:560.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:35:560.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:560.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692275560528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:560.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:767.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:767.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692275767618 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:767.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:767.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:767.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692275767793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42921 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:767.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:768.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:35:768.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:35:768.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:768.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275768097 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:768.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:780.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:781.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:35:781.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:35:781.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275780780 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:781.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:781.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:781.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275780901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41503 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:781.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:781.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:781.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275781191 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:781.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:781.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:782.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275781317 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:782.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:782.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:782.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275781398 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:35:782.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:35:782.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:782.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275781541 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:35:782.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:35:782.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:782.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275781558 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:35:782.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:35:782.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:782.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275782136 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:782.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:782.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:782.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275782163 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:35:782.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:35:782.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:35:829.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:829.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275829243 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:829.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:829.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:829.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692275829680 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34159 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:829.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:830.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:35:830.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:35:830.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:830.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692275830184 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:830.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:941.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:941.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692275941642 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:941.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:942.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:942.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692275942134 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42403 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:942.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:942.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:35:942.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:35:942.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:942.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692275942559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:942.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:622.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:622.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692276622729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:622.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:623.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:623.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692276623221 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36963 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:623.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:623.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:36:623.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:36:623.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:623.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692276623686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:623.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:730.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:730.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692276730320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:730.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:730.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:730.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692276730503 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33939 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:730.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:730.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:36:730.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:36:730.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:731.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692276730964 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:731.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:567.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:567.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692278567089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:567.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:567.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:567.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692278567303 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44575 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:567.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:567.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:38:567.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:38:567.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:567.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692278567793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:567.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:657.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:657.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692278657797 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:657.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:658.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:658.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692278658715 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46041 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:658.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:658.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:38:659.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:38:659.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:659.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692278659045 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:659.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:656.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:656.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692279656327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:656.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:656.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:656.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692279656866 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43647 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:656.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:657.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:39:657.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:39:657.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:657.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692279657281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:657.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:657.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:657.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692279657290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:657.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:657.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:657.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692279657370 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:39:657.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:39:657.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:657.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692279657439 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:39:657.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:39:658.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:658.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692279658263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:658.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:658.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:658.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692279658318 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:39:658.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:39:658.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:39:707.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:707.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692279707541 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:707.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:708.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:708.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692279708321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46471 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:708.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:708.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:39:708.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:39:708.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:708.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692279708866 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:708.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:866.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:866.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692279866639 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:866.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:867.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:867.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692279867138 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33421 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:867.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:867.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:39:867.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:39:867.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:867.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692279867577 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:867.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:867.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:867.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692279867584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:867.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:867.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:867.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692279867669 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:39:867.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:39:867.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:867.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692279867762 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:39:867.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:39:867.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:867.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692279867783 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:39:867.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:39:868.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:868.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692279868525 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:868.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:868.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:868.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:39:868.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692279868572 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:39:868.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:39:914.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:914.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692279914781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:914.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:915.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:915.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692279915357 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38635 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:915.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:915.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:39:915.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:39:915.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:915.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692279915654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:915.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:40:654.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:40:654.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692280654024 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:40:654.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:40:654.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:40:654.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692280654647 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44695 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:40:654.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:40:655.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:40:655.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:40:655.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:40:655.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692280655178 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:40:655.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:40:941.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:40:941.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692280941821 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:40:941.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:40:942.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:40:942.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692280942201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42245 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:40:942.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:40:942.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:40:942.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:40:942.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:40:942.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692280942576 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:40:942.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:295.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:295.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692282295244 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:295.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:295.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:295.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692282295443 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33181 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:295.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:295.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:42:295.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:42:295.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:296.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692282295907 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:296.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:296.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:296.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:42:296.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692282295912 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:296.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:296.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:296.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692282295967 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:42:296.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:42:296.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:296.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692282296018 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:42:296.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:42:296.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:296.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692282296554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:296.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:296.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:296.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692282296592 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:42:296.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:42:341.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:341.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692282341719 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:341.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:342.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:342.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692282342350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33905 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:342.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:342.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:42:342.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:42:342.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:342.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692282342702 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:342.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:671.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:672.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692282671899 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:672.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:672.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:672.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692282672373 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35365 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:672.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:672.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:42:673.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:42:673.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:673.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692282673049 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:673.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:673.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:673.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692282673054 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:673.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:673.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:673.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692282673124 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:42:673.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:42:673.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:673.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692282673245 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:42:673.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:42:673.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:673.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692282673263 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:42:673.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:42:673.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:673.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692282673872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:673.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:673.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:673.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:42:673.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692282673923 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:42:674.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:42:689.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:689.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692282689320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:689.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:689.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:689.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692282689647 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35591 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:689.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:689.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:42:690.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:42:690.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:690.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692282690044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:690.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:721.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:721.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692282721170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:721.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:721.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:721.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692282721621 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35519 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:721.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:721.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:42:722.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:42:722.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:722.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692282722144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:722.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:958.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:959.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692282958766 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:959.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:959.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:959.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692282958960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33533 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:959.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:42:959.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:42:959.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:42:959.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:42:959.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692282959307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:42:959.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:251.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:252.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692283251859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:252.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:252.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:252.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692283252039 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35319 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:252.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:252.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:43:252.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:43:252.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:252.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692283252407 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:252.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:685.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:685.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692283685337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:685.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:685.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:685.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692283685507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33415 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:685.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:685.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:43:685.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:43:685.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:685.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692283685938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:685.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:322.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:323.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:45:323.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:45:323.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692285322616 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:323.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:323.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:323.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692285322805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34197 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:323.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:323.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:323.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692285323150 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:323.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:576.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:576.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692285576033 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:576.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:576.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:576.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692285576589 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36231 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:576.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:576.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:45:577.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:45:577.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:577.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692285577142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:577.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:866.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:866.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692286866193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:866.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:866.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:866.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692286866673 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44569 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:866.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:867.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:46:867.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:46:867.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:867.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692286867173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:867.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:867.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:867.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692286867181 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:867.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:867.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:867.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692286867271 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:46:867.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:46:867.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:867.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692286867324 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:46:867.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:46:867.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:867.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692286867904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:867.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:867.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:867.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:46:867.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692286867936 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:46:868.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:46:913.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:913.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692286913157 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:913.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:913.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:913.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692286913345 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45297 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:913.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:913.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:46:913.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:46:913.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:913.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692286913708 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:913.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:027.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:027.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692287027513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:027.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:028.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:028.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692287028109 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39341 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:028.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:028.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:47:028.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:47:028.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:028.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692287028452 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:028.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:028.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:028.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692287028467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:028.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:028.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:028.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692287028620 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:47:028.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:47:028.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:028.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692287028740 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:47:028.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:47:028.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:028.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692287028769 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:47:028.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:47:029.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:029.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692287029328 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:029.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:029.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:029.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692287029367 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:47:029.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:47:029.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:47:078.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:078.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692287078477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:078.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:078.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:078.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692287078675 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41611 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:078.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:079.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:47:079.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:47:079.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:079.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692287079103 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:079.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:957.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:958.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692287957849 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:958.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:958.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:958.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692287958035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43273 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:958.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:958.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:47:958.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:47:958.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:958.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692287958410 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:958.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:997.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:997.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692287997280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:997.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:997.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:997.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692287997455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38539 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:997.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:47:997.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:47:997.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:47:997.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:47:997.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692287997905 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:47:997.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:499.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:499.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692289499304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:499.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:499.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:49:499.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:49:500.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:49:500.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:500.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692289499501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44451 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:500.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:500.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:500.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692289499842 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:500.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:500.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:500.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692289499846 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:500.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:500.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:500.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692289499908 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:49:500.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:500.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:500.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692289499969 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:49:500.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:500.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:500.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692289500498 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:500.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:500.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:500.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692289500552 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:49:500.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:550.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:550.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692289550684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:550.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:550.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:550.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692289550893 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42991 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:550.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:551.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:49:551.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:49:551.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:551.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692289551401 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:551.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:789.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:789.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692289789301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:789.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:789.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:789.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692289789866 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36813 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:789.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:790.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:49:790.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:49:790.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:790.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692289790355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:790.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:790.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:790.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692289790363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:790.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:790.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:790.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692289790441 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:49:790.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:790.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:790.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692289790532 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:49:790.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:790.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:790.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692289790550 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:49:790.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:791.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:791.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692289791093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:791.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:791.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:791.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692289791132 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:49:791.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:791.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:49:843.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:843.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692289843284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:843.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:843.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:843.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692289843497 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45975 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:843.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:843.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:49:843.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:49:843.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:843.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692289843816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:843.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:064.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:064.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692290064608 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:064.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:064.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:064.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692290064786 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41125 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:064.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:065.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:50:065.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:50:065.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:065.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692290065157 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:065.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:068.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:068.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692290067998 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:068.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:068.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:068.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692290068089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41381 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:068.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:068.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:50:068.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:50:068.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:068.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692290068483 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:068.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:505.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:505.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692290505687 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:505.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:506.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:506.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692290506284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44515 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:506.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:506.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:50:506.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:50:506.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:506.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692290506709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:506.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:752.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:753.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692290752931 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:753.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:753.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:753.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692290753566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42965 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:753.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:753.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:50:753.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:50:753.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:753.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692290753848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:753.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:522.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:522.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692292522790 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:522.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:523.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:523.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692292523006 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43811 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:523.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:523.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:52:523.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:52:523.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:523.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692292523415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:523.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:626.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:626.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692292626027 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:626.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:626.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:626.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692292626626 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39053 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:626.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:626.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:52:626.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:52:626.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:626.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692292626921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:626.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:773.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:773.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692293773309 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:773.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:774.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:774.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692293773931 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43901 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:774.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:774.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:53:774.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:53:774.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:774.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692293774568 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:774.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:774.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:774.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692293774573 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:774.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:774.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:774.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692293774626 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:53:774.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:53:774.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:774.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692293774675 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:53:774.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:53:775.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:775.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692293775194 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:775.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:775.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:775.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692293775235 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:53:775.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:53:775.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:53:776.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:776.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692293776540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:776.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:777.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:777.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692293777000 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45657 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:777.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:777.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:53:777.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:53:777.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:777.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692293777369 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:777.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:777.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:777.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692293777377 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:777.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:777.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:777.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692293777452 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:53:777.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:53:777.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:777.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692293777569 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:53:777.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:53:777.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:777.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692293777593 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:53:777.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:53:778.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:778.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692293778109 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:778.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:778.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:778.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:53:778.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692293778207 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:53:778.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:53:822.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:822.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692293822573 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:822.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:823.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:823.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692293823079 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40073 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:823.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:823.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:53:823.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:53:823.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:823.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692293823449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:823.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:831.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:831.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692293831345 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:831.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:831.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:831.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692293831516 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34233 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:831.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:831.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:53:831.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:53:831.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:831.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692293831889 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:831.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:54:759.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:759.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692294759010 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:54:759.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:54:759.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:759.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692294759212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33761 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:54:759.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:54:759.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:54:759.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:54:759.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:759.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692294759618 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:54:759.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:54:885.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:885.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692294885554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:54:885.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:54:885.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:885.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692294885747 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35431 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:54:885.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:54:886.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:54:886.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:54:886.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:886.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692294886234 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:54:886.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:159.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:159.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692296159557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:159.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:159.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:159.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692296159738 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42223 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:160.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:160.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:56:160.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:160.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:160.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692296160343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:160.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:160.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:160.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692296160349 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:160.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:160.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:160.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692296160445 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:160.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:160.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:160.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692296160497 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:56:160.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:161.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:161.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692296161016 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:161.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:161.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:161.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:56:161.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692296161040 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:161.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:206.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:206.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692296206198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:206.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:206.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:206.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692296206663 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46789 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:206.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:206.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:56:207.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:207.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:207.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692296207140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:207.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:299.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:299.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692296299159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:299.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:299.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:299.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692296299679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35451 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:299.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:299.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:56:300.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:300.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:300.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692296300038 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:300.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:300.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:300.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692296300044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:300.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:300.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:300.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692296300139 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:300.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:300.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:300.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692296300241 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:300.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:300.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:300.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692296300258 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:56:300.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:300.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:56:301.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:301.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692296300795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:301.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:301.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:301.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692296300841 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:301.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:350.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:350.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692296350012 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:350.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:350.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:350.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692296350360 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34703 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:350.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:350.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:56:350.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:350.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:350.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692296350860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:350.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:775.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:775.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692296775826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:775.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:776.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:776.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692296776484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38971 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:776.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:776.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:56:776.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:776.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:776.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692296776897 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:776.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:794.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:794.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692296794372 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:794.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:794.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:794.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692296794943 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35953 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:794.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:795.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:56:795.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:795.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:795.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692296795483 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:795.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:215.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:215.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692297215314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:215.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:215.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:215.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692297215877 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34387 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:215.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:216.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:57:216.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:57:216.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:216.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692297216345 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:216.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:278.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:278.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692297278031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:278.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:278.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:278.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692297278273 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35829 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:278.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:278.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:57:278.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:57:278.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:278.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692297278719 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:278.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:088.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:088.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692299088509 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:088.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:088.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:088.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692299088693 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:088.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:089.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:59:089.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:59:089.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:089.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692299089451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:089.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:384.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:385.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692299384897 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:385.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:385.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:385.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692299385494 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33167 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:385.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:385.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:59:385.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:59:385.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:385.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692299385938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:385.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:431.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:431.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692300431681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:431.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:431.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:431.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692300431887 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42073 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:431.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:432.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:00:432.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:00:432.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:432.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692300432217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:432.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:432.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:432.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692300432223 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:432.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:432.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:432.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692300432318 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:00:432.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:00:432.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:432.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692300432368 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:00:432.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:00:432.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:432.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692300432901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:432.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:00:432.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:433.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:433.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692300432947 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:00:433.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:00:485.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:485.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692300485154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:485.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:485.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:485.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692300485334 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45375 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:485.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:485.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:00:485.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:00:485.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:485.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692300485780 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:485.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:665.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:665.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692300665029 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:665.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:665.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:665.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692300665230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40809 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:665.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:665.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:00:665.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:00:665.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:665.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692300665813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:665.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:665.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:665.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692300665827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:665.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:665.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:666.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692300665903 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:00:666.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:00:666.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:666.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692300666320 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:00:666.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:00:666.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:666.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692300666340 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:00:666.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:00:666.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:666.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692300666945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:666.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:667.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:667.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:00:667.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692300667011 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:00:667.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:00:713.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:713.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692300713209 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:713.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:713.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:713.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692300713387 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46781 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:713.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:713.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:00:713.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:00:713.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:713.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692300713681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:713.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:503.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:503.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692301502980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:503.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:503.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:503.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692301503212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:503.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:503.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:01:503.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:01:503.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:503.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692301503797 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:503.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:784.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:784.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692301784852 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:784.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:785.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:785.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692301785066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39095 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:785.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:785.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:01:785.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:01:785.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:785.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692301785475 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:785.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:829.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:830.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692301829751 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 1909 dequeues = 1909 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 1940 dequeues = 1940 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692272846570 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692301785488 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:05:01:830.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:01:830.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:830.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692301829894 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 382 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 433 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 102 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1462 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 418 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 956, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1909, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:01:830.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:01:838.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:838.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692301838353 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.25, ] ] ] 2025-09-01 Mon 02:05:01:838.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:05:01:838.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:838.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692301838401 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1375572704 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.04 five_min_avg = 0.17 fifteen_min_avg = 0.165 ] cpu_share = 1.78333 ] ] 2025-09-01 Mon 02:05:01:838.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:01:838.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:838.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692301838408 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.78333 used_sys_mem = 2854240503 one_min_cpuload = 0.04 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:05:01:838.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:05:01:838.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:838.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692301838434 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 579 in_bytes = 59712 out_tpkts = 618 out_bytes = 34195 exception_packets = 32 exception_packets_allowed = 21 raw_vhost_stats= [ name = vhost0 in_pkts = 579 in_bytes = 59712 out_pkts = 618 out_bytes = 34195 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 527 in_bytes = 30373 out_pkts = 582 out_bytes = 59838 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:05:01:838.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:01:838.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:838.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692301838460 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:05:01:838.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:05:02:846.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:05:02:847.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:05:02:847.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:02:847.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692302846767 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:02:847.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:02:847.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:02:847.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:05:02:847.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692302847439 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 382 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 433 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 102 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1462 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 418 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 956, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1916, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:02:847.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:02:847.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:05:02:847.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:05:02:847.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:05:02:848.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:02:848.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Idle 2025-09-01 Mon 02:05:02:848.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:02:848.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692302847951 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 383 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 433 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 102 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1462 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 418 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 956, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1918, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:02:848.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:02:848.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:02:848.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692302847987 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:02:848.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:02:848.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:05:02:848.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:02:848.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692302848088 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 383 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 434 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 102 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1462 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 418 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 957, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1918, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:02:848.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:03:013.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:013.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692303013669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:013.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:013.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:013.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692303013902 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45511 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:013.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:014.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:014.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:014.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:014.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692303014418 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:014.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:014.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:014.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692303014426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:014.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:014.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:014.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692303014517 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:014.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:014.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:014.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692303014584 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:03:014.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:015.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:015.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692303015093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:015.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:015.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:015.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:03:015.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692303015154 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:015.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:064.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:064.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692303064336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:064.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:064.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:064.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692303064897 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43681 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:064.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:065.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:065.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:065.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:065.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692303065234 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:065.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:202.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:202.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692303202057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:202.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:202.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:202.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692303202724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33505 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:202.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:203.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:203.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:203.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:203.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692303203139 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:203.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:203.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:203.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692303203148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:203.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:203.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:203.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692303203224 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:203.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:203.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:203.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692303203315 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:203.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:203.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:203.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692303203331 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:03:203.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:204.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:204.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692303203951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:204.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:204.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:204.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692303204016 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:204.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:204.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:03:253.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:253.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692303253137 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:253.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:253.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:253.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692303253774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34291 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:253.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:254.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:254.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:254.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:254.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692303254339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:254.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:375.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:376.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692303375933 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:376.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:376.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:376.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692303376391 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40623 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:376.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:376.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:376.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:376.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:377.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692303376951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:377.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:711.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:711.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692303711681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:712.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:712.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:712.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692303711907 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45701 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:712.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:712.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:712.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:712.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:712.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692303712278 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:712.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:001.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:001.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692304001344 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:001.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:001.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:002.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692304001956 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:002.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:002.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:04:002.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:04:002.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:002.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692304002399 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:002.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:154.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:154.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692304154494 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:154.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:155.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:155.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692304155190 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41541 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:155.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:155.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:04:155.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:04:155.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:155.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692304155686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:155.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:090.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:090.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692306090636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:090.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:091.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:091.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692306091282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40187 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:091.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:091.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:06:091.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:06:091.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:091.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692306091668 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:091.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:207.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:207.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692306207868 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:207.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:208.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:208.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692306208113 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45767 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:208.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:208.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:06:208.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:06:208.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:208.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692306208621 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:208.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:977.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:977.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692306977077 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:977.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:977.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:977.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692306977600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39747 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:977.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:977.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:06:978.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:06:978.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:978.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692306978088 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:978.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:978.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:978.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692306978099 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:978.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:978.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:978.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692306978206 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:06:978.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:06:978.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:978.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692306978257 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:06:978.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:06:978.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:978.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692306978752 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:978.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:978.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:978.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:06:978.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692306978832 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:06:979.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:07:030.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:030.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692307030005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:07:030.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:07:030.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:030.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692307030572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38409 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:07:030.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:07:030.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:07:030.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:07:030.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:031.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692307030967 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:07:031.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:07:744.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:744.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692307744494 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:07:744.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:07:744.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:07:744.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:07:745.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:745.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692307744678 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40319 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:07:745.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:07:745.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:745.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692307745016 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:07:745.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:07:745.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:745.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692307745020 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:07:745.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:07:745.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:745.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692307745086 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:07:745.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:07:745.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:745.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692307745446 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:07:745.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:07:745.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:745.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692307745468 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:07:745.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:07:746.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:746.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:07:746.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692307746455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:07:746.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:07:746.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:746.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692307746495 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:07:746.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:07:798.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:798.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692307798638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:07:798.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:07:799.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:799.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692307799251 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43199 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:07:799.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:07:799.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:07:799.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:07:799.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:799.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692307799764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:07:799.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:08:003.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:003.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692308003041 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:08:003.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:08:003.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:003.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692308003234 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34981 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:08:003.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:08:003.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:08:003.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:08:003.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:003.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692308003709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:08:003.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:08:780.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:781.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692308780927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:08:781.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:08:781.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:781.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692308781554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40239 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:08:781.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:08:781.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:08:782.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:08:782.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:782.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692308782302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:08:782.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:019.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:020.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692310019931 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:020.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:020.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:020.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692310020462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44433 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:020.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:020.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:020.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:020.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:020.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692310020907 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:020.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:195.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:195.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692310195837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:195.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:196.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:196.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692310196435 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36095 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:196.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:196.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:196.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:196.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:196.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692310196782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:196.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:196.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:196.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692310196788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:196.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:196.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:196.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692310196845 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:196.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:196.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:196.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692310196897 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:10:196.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:197.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:197.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692310197461 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:197.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:197.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:197.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692310197498 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:197.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:197.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:10:204.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:204.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692310204709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:204.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:205.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:205.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692310205213 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44571 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:205.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:205.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:205.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:205.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:205.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692310205598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:205.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:205.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:205.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692310205603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:205.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:205.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:205.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692310205677 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:205.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:205.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:205.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692310205764 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:205.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:205.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:205.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692310205780 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:10:205.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:206.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:206.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692310206301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:206.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:206.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:206.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:10:206.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692310206339 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:206.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:246.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:246.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692310246659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:246.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:247.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:247.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692310247249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37973 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:247.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:247.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:247.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:247.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:247.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692310247721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:247.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:254.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:254.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692310254414 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:254.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:254.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:254.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692310254786 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40657 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:254.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:255.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:255.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:255.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:255.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692310255219 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:255.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:744.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:744.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692310744470 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:744.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:744.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:744.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692310744688 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44533 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:744.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:745.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:745.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:745.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:745.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692310745110 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:745.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:182.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:182.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692311182360 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:182.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:182.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:182.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692311182794 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41095 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:182.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:182.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:11:183.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:11:183.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:183.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692311183111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:183.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:318.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:318.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692311318886 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:319.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:319.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:319.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692311319352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41353 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:319.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:319.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:11:320.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:11:320.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:320.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692311320098 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:320.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:307.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:307.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692313307300 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:307.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:307.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:307.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692313307892 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46329 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:308.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:308.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:13:308.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:13:308.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:308.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692313308208 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:308.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:318.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:318.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692313318230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:318.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:318.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:318.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692313318859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46741 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:318.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:319.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:13:319.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:13:319.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:319.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692313319694 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:319.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:981.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:981.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692313980990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:981.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:981.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:981.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692313981467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33307 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:981.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:981.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:13:981.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:13:981.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:982.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692313981977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:982.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:982.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:982.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692313981986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:982.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:982.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:982.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692313982085 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:13:982.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:13:982.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:982.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692313982171 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:13:982.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:13:982.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:982.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692313982697 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:982.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:982.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:982.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:13:982.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692313982726 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:13:982.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:14:032.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:033.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692314032912 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:14:033.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:14:033.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:14:033.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:033.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692314033084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34175 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:14:033.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:14:033.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:14:033.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:033.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692314033754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:14:033.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:14:813.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:813.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692314813269 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:14:813.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:14:813.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:813.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692314813490 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34517 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:14:813.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:14:813.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:14:813.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:14:814.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:814.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692314814005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:14:814.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:14:814.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:814.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692314814011 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:14:814.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:14:814.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:814.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692314814098 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:14:814.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:14:814.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:814.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692314814354 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:14:814.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:14:814.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:814.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692314814389 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:14:814.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:14:814.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:814.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692314814948 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:14:815.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:14:815.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:815.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:14:815.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692314814976 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:14:815.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:14:867.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:867.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692314867142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:14:867.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:14:867.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:867.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692314867317 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41851 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:14:867.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:14:867.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:14:867.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:14:867.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:867.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692314867728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:14:867.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:15:077.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:15:078.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692315077950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:15:078.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:15:078.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:15:078.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692315078159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:15:078.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:15:078.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:15:078.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:15:078.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:15:078.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692315078522 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:15:078.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:15:121.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:40970 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:15:130.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:40978 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:15:848.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:15:848.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692315848843 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:15:848.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:15:849.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:15:849.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692315849336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35035 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:15:849.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:15:849.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:15:849.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:15:849.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:15:849.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692315849795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:15:849.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:16:968.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:16:969.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:16:969.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:16:969.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692316968729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:16:969.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:16:969.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:16:969.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692316968932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41053 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:16:969.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:16:969.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:16:969.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692316969251 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:16:969.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:304.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:304.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692317304393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:304.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:304.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:304.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692317304578 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36391 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:304.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:304.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:17:304.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:17:304.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:305.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692317304890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:305.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:305.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:305.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692317304895 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:305.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:305.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:305.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692317304957 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:305.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:305.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:305.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692317305205 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:305.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:305.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:305.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692317305225 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:17:305.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:306.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:306.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692317305975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:306.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:306.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:306.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:17:306.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692317306017 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:306.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:355.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:355.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692317355227 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:355.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:355.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:355.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692317355417 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43517 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:355.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:355.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:17:355.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:17:355.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:355.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692317355950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:355.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:495.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:495.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692317495855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:495.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:496.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:496.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692317496332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46677 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:496.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:496.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:17:496.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:17:496.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:496.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692317496918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:496.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:496.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:496.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692317496926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:496.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:496.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:497.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692317496987 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:497.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:497.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:497.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692317497035 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:17:497.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:497.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:497.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692317497641 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:497.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:497.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:17:497.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:497.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692317497680 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:497.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:548.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:549.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692317548854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:549.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:549.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:549.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692317549035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33229 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:549.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:549.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:17:549.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:17:549.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:549.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692317549417 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:549.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:793.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:794.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692317793948 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:794.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:794.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:794.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692317794464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46777 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:794.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:794.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:17:794.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:17:794.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:794.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692317794921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:794.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:310.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:310.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692318310106 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:310.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:310.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:310.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692318310312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43777 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:310.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:310.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:18:310.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:18:310.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:310.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692318310755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:310.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:548.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:548.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692318548572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:548.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:549.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:549.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692318549093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38429 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:549.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:549.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:18:549.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:18:549.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:549.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692318549583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:549.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:129.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:129.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692320128905 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:129.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:129.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:129.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692320129471 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46085 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:129.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:129.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:20:130.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:20:130.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:130.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692320130216 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:130.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:511.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:511.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692320511787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:511.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:512.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:512.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692320512333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44607 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:512.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:512.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:20:512.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:20:512.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:512.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692320512826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:512.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:965.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:965.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692320965384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:965.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:965.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:966.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692320965950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38573 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:966.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:966.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:20:966.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:20:966.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:966.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692320966225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:966.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:966.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:966.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692320966232 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:966.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:966.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:966.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692320966399 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:20:966.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:20:966.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:966.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692320966459 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:20:966.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:20:966.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:967.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692320966953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:967.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:967.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:967.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:20:967.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692320967000 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:20:967.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:21:012.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:012.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692321012198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:012.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:012.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:012.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692321012364 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45337 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:012.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:012.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:21:012.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:21:012.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:012.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692321012743 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:012.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:431.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:431.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692321431058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:431.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:431.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:431.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692321431273 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43465 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:431.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:431.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:21:431.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:21:431.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:431.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692321431667 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:431.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:431.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:431.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692321431673 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:431.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:431.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:431.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692321431750 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:21:431.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:21:431.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:431.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692321431857 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:21:431.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:21:431.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:431.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692321431877 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:21:431.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:21:432.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:432.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:21:432.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692321432424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:432.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:432.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:432.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692321432464 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:21:432.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:21:479.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:479.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692321479622 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:479.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:480.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:480.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692321480338 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35431 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:480.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:480.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:21:481.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:21:481.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:481.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692321481224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:481.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:912.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:913.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692321912905 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:913.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:913.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:913.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692321913095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41329 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:913.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:21:913.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:21:913.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:21:913.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:21:913.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692321913589 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:21:913.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:22:382.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:22:382.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692322382045 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:22:382.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:22:382.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:22:382.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692322382527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39157 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:22:382.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:22:382.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:22:382.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:22:382.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:22:382.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692322382909 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:22:382.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:23:802.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:802.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692323802392 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:23:802.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:23:803.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:803.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692323802977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42767 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:23:803.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:23:803.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:23:803.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:23:803.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:803.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692323803435 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:23:803.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:23:803.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:803.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692323803442 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:23:803.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:23:803.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:803.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692323803535 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:23:803.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:23:803.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:803.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692323803634 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:23:803.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:23:803.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:803.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692323803637 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:23:803.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:23:803.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:803.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692323803660 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:23:803.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:23:803.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:803.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692323803711 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34497 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:23:803.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:23:803.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:23:803.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:23:803.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:803.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692323803948 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:23:804.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:23:804.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:804.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692323804228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:23:804.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:23:804.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:804.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692323804260 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:23:804.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:23:804.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:23:850.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:850.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692323850404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:23:850.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:23:850.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:850.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692323850823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39909 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:23:850.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:23:851.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:23:851.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:23:851.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:23:851.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692323851305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:23:851.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:327.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:327.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692324327041 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:327.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:327.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:327.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692324327260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35665 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:327.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:327.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:24:327.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:24:327.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:327.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692324327687 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:327.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:545.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:545.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692324544994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:545.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:545.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:545.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692324545509 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41881 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:545.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:545.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:24:545.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:24:546.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:546.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692324546019 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:546.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:546.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:546.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692324546024 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:546.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:546.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:546.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692324546082 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:24:546.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:546.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:546.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692324546160 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:24:546.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:546.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:546.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692324546710 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:546.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:546.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:24:546.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:546.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692324546751 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:24:546.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:596.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:596.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692324596861 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:596.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:597.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:597.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692324597397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40657 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:597.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:597.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:24:597.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:24:597.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:597.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692324597744 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:597.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:832.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:832.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692324832465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:832.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:832.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:832.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692324832655 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35029 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:832.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:832.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:24:833.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:24:833.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:833.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692324833065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:833.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:596.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:596.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692325596845 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:596.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:597.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:597.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692325597469 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39573 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:597.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:597.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:25:597.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:25:597.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:597.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692325597767 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:597.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:26:831.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:831.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692326831197 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:26:831.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:26:831.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:831.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692326831668 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33721 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:26:831.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:26:831.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:26:832.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:26:832.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:832.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692326832104 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:26:832.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:397.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:398.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692327397924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:398.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:398.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:398.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692327398480 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46667 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:398.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:398.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:27:398.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:27:398.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:398.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692327398826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:398.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:620.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:620.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692327620078 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:620.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:620.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:620.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692327620630 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36565 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:620.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:621.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:27:621.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:27:621.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:621.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692327621149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:621.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:621.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:621.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692327621158 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:621.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:621.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:621.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692327621253 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:27:621.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:27:621.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:621.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692327621299 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:27:621.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:27:622.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:622.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692327622229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:622.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:622.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:622.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:27:622.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692327622263 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:27:622.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:27:667.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:667.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692327667456 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:667.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:667.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:667.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692327667628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39947 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:667.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:667.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:27:668.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:27:668.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:668.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692327668519 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:668.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:28:035.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:035.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692328035835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:28:035.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:28:036.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:036.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692328036277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41425 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:28:036.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:28:036.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:28:036.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:28:036.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:036.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692328036651 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:28:036.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:28:036.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:036.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692328036657 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:28:036.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:28:036.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:036.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692328036720 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:28:036.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:28:036.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:036.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692328036827 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:28:036.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:28:036.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:036.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692328036844 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:28:036.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:28:037.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:037.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692328037547 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:28:037.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:28:037.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:037.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:28:037.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692328037595 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:28:038.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:28:086.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:086.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692328086716 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:28:086.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:28:086.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:086.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692328086887 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33047 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:28:086.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:28:087.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:28:087.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:28:087.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:087.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692328087365 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:28:087.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:28:631.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:631.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692328631657 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:28:631.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:28:632.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:632.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692328631855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39299 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:28:632.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:28:632.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:28:632.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:28:632.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:28:632.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692328632257 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:28:632.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:29:113.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:29:113.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692329113514 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:29:113.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:29:114.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:29:114.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692329114032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36851 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:29:114.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:29:114.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:29:114.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:29:114.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:29:114.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692329114540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:29:114.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:522.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:522.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692330522445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:522.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:522.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:522.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692330522650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32839 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:522.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:522.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:30:523.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:30:523.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:523.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692330523063 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:523.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:720.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:720.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692330720250 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:720.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:720.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:720.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692330720813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38929 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:720.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:721.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:30:721.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:30:721.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:721.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692330721198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:721.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:721.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:721.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692330721206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:721.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:721.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:721.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692330721309 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:30:721.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:30:721.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:721.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692330721403 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:30:721.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:30:721.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:721.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692330721422 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:30:721.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:30:722.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:722.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692330721961 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:722.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:722.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:722.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:30:722.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692330722024 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:30:722.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:30:769.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:769.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692330769174 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:769.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:769.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:769.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692330769355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46475 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:769.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:769.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:30:769.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:30:769.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:769.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692330769694 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:769.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:184.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:184.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692331184721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:184.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:185.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:185.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692331185234 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46415 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:185.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:185.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:31:185.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:31:185.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:185.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692331185717 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:185.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:503.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:503.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:31:503.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:31:503.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692331503045 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:503.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:503.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:503.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692331503311 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46295 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:503.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:503.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:503.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692331503674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:503.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:503.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:503.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692331503679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:503.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:503.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:503.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692331503756 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:31:503.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:31:503.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:503.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692331503809 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:31:503.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:31:504.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:504.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:31:504.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692331504595 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:504.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:504.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:504.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692331504638 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:31:504.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:31:553.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:553.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692331553770 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:553.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:554.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:554.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692331554234 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37185 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:554.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:554.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:31:554.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:31:554.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:554.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692331554734 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:554.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:687.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:687.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692331687894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:688.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:688.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:688.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692331688148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38253 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:688.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:688.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:31:688.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:31:688.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:688.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692331688460 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:688.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:830.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:830.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692331830054 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 2233 dequeues = 2233 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 2267 dequeues = 2267 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692302848049 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692331688470 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:05:31:830.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:31:830.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:830.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692331830187 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 447 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 507 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 120 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1711 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 489 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1119, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2233, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:31:830.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:31:838.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:838.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692331838563 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:05:31:838.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:05:31:838.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:838.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692331838604 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 683 in_bytes = 77064 out_tpkts = 727 out_bytes = 40509 exception_packets = 34 exception_packets_allowed = 23 raw_vhost_stats= [ name = vhost0 in_pkts = 683 in_bytes = 77064 out_pkts = 727 out_bytes = 40509 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 7.11237e-15 total_out_bandwidth_utilization = 1.69732e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1312, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 3131, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 621 in_bytes = 36057 out_pkts = 686 out_bytes = 77190 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:05:31:838.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:31:838.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:838.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692331838627 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:05:31:838.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:05:32:598.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:599.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692332598917 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:32:599.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:32:599.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:599.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692332599451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38183 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:32:599.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:32:599.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:32:599.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:32:599.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:599.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692332599927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:32:599.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:32:848.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:05:32:848.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:05:32:848.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:32:848.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692332848311 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:32:848.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:32:848.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:32:849.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692332848887 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 447 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 508 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 120 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1714 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 490 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1121, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2241, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:32:849.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:32:849.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:05:32:849.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:05:32:849.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:05:32:849.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:05:32:849.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:32:849.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Idle 2025-09-01 Mon 02:05:32:849.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:849.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692332849374 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 448 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 508 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 120 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1714 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 490 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1121, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2243, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:32:849.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:32:849.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:849.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692332849418 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:32:849.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:32:849.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:05:32:849.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:849.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692332849556 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 448 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 509 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 120 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1714 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 490 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1122, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2243, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:32:849.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:33:596.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:596.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692333596656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:596.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:596.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:596.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692333596871 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36621 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:596.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:597.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:33:597.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:33:597.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:597.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692333597212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:597.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:195.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:195.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692334195293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:195.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:196.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:196.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692334196019 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38395 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:196.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:196.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:34:196.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:34:196.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:197.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:34:197.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692334196479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:197.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:197.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:197.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692334196493 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:197.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:197.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:197.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692334196571 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:34:198.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:34:198.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:198.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692334196624 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:34:198.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:34:198.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:198.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692334197180 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:198.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:198.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:198.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692334197220 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:34:198.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:34:246.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:246.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692334246346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:246.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:247.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:247.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692334246939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36139 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:247.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:247.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:34:247.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:34:247.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:247.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692334247267 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:247.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:706.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:706.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692334706109 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:706.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:706.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:706.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692334706333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35419 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:706.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:706.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:34:706.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:34:706.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:706.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692334706736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:706.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:894.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:894.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692334894364 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:894.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:895.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:895.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692334895182 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40457 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:895.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:895.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:34:895.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:34:895.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:895.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692334895682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:895.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:895.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:895.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692334895689 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:895.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:895.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:895.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692334895769 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:34:895.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:34:895.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:895.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692334895917 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:34:895.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:34:895.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:896.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692334895951 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:34:896.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:34:896.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:896.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692334896792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:896.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:896.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:896.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692334896843 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:34:896.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:34:897.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:34:945.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:945.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692334944965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:945.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:945.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:945.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692334945403 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44099 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:945.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:945.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:34:945.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:34:945.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:945.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692334945901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:945.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:35:201.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:35:201.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692335201447 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:35:201.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:35:201.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:35:202.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692335201980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:35:202.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:35:202.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:35:202.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:35:202.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:35:202.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692335202345 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:35:202.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:35:882.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:35:882.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692335882003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:35:882.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:35:882.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:35:882.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692335882607 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45763 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:35:882.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:35:882.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:35:882.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:35:883.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:35:883.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692335882982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:35:883.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:020.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:020.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692337020479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:020.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:020.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:020.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692337020677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36777 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:020.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:020.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:37:020.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:37:020.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:021.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692337020977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:021.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:269.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:269.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692337269387 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:269.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:270.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:270.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692337270120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44889 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:270.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:270.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:37:270.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:37:270.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:270.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692337270494 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:270.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:270.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:270.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692337270500 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:270.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:270.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:270.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692337270570 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:37:270.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:37:271.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:271.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692337270992 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:37:271.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:37:271.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:271.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692337271021 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:37:271.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:37:271.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:271.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692337271597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:271.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:271.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:271.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692337271625 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:37:271.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:37:271.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:37:323.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:323.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692337323851 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:324.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:324.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:324.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692337324080 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34857 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:324.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:324.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:37:324.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:37:324.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:324.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692337324487 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:324.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:809.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:809.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692337809174 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:809.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:809.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:809.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692337809707 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46791 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:809.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:810.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:37:810.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:37:810.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:810.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692337810267 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:810.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:314.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:314.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692338314611 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:314.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:315.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:315.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692338315202 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38391 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:315.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:315.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:38:315.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:38:315.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:315.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692338315708 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:315.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:810.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:810.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692338810874 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:811.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:811.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:811.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692338811467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40409 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:811.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:811.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:38:811.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:38:812.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:38:812.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:812.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692338811973 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:812.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:812.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:812.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692338811983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:812.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:812.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:812.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692338812044 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:38:812.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:38:812.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:812.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692338812102 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:38:812.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:38:812.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:812.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692338812625 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:812.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:812.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:812.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692338812680 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:38:812.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:38:857.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:857.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692338857810 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:857.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:857.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:858.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692338857987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45797 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:858.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:858.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:38:858.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:38:858.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:858.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692338858381 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:858.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:39:866.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:39:866.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692339866529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:39:866.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:39:867.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:39:867.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692339867073 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34497 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:39:867.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:39:867.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:39:867.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:39:867.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:39:867.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692339867765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:39:867.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:404.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:404.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692340403843 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:404.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:404.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:404.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692340403994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45261 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:404.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:404.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:40:404.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:40:404.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:404.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692340404425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:404.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:909.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:909.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692340909225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:909.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:909.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:910.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692340909933 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34853 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:910.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:910.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:40:910.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:40:910.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:910.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692340910816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:910.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:910.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:910.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692340910823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:910.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:910.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:910.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692340910918 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:40:910.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:40:911.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:911.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692340910980 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:40:911.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:40:911.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:911.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692340911575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:911.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:911.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:40:911.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:911.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692340911613 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:40:911.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:40:961.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:961.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692340961759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:961.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:961.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:962.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692340961983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33721 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:962.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:962.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:40:962.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:40:962.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:962.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692340962424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:962.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:698.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:698.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692341698455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:698.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:698.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:698.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692341698638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34869 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:698.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:698.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:41:698.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:41:699.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:699.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692341699058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:699.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:699.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:699.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692341699064 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:699.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:699.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:699.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692341699162 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:41:699.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:41:700.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:700.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692341700374 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:41:700.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:41:700.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:700.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692341700400 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:41:700.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:41:701.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:701.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692341701473 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:701.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:701.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:701.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:41:701.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692341701583 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:41:701.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:41:752.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:752.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692341752682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:752.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:753.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:753.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692341753238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33511 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:753.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:753.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:41:753.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:41:753.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:753.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692341753608 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:753.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:793.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:794.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692341793971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:794.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:794.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:794.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692341794202 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45681 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:794.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:794.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:41:794.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:41:794.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:794.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692341794608 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:794.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:988.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:988.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692341988646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:988.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:989.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:989.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692341989240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39139 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:989.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:989.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:41:989.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:41:989.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:989.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692341989696 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:989.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:42:734.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:42:735.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692342734776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:42:735.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:42:735.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:42:735.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692342734954 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41901 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:42:735.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:42:735.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:42:735.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:42:735.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:42:735.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692342735281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:42:735.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:43:933.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:43:934.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692343933881 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:43:934.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:43:934.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:43:934.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692343934692 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43991 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:43:934.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:43:935.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:43:935.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:43:935.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:43:935.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692343935693 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:43:935.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:112.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:112.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692344112574 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:112.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:112.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:112.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692344112772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37285 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:112.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:113.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:44:113.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:44:113.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:113.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692344113278 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:113.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:113.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:113.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692344113284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:113.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:113.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:113.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692344113383 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:44:114.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:114.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:114.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692344113501 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:44:114.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:114.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:114.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692344113539 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:44:114.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:114.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:114.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692344114119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:114.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:114.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:114.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692344114174 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:44:114.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:114.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:44:164.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:164.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692344164286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:164.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:164.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:164.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692344164465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38445 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:164.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:164.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:44:164.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:44:164.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:164.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692344164881 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:164.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:643.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:643.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692344643476 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:643.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:644.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:644.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692344644257 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35267 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:644.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:644.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:44:644.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:44:644.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:644.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692344644566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:644.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:155.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:155.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692345155052 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:155.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:155.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:155.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692345155298 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34305 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:155.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:155.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:45:155.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:45:155.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:155.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692345155756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:155.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:898.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:898.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692345898728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:898.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:899.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:899.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692345899159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39711 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:899.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:899.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:45:899.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:45:899.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:899.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692345899612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:899.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:899.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:899.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692345899618 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:899.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:899.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:899.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692345899726 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:45:899.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:45:899.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:899.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692345899802 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:45:899.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:45:900.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:900.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692345900355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:900.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:900.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:900.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:45:900.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692345900382 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:45:900.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:45:948.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:948.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692345948519 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:948.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:949.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:949.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692345949084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34725 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:949.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:949.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:45:949.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:45:949.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:949.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692345949392 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:949.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:46:867.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:46:867.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692346867511 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:46:867.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:46:867.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:46:867.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692346867705 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37663 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:46:867.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:46:867.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:46:868.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:46:868.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:46:868.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692346868050 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:46:868.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:226.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:226.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692347226792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:226.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:227.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:227.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692347227276 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40809 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:227.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:227.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:47:227.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:47:227.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:227.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692347227835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:227.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:679.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:680.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692347679898 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:680.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:680.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:680.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692347680550 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33735 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:680.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:680.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:47:680.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:47:680.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:681.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692347680968 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:681.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:681.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:681.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692347680974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:681.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:681.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:681.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692347681086 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:47:681.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:47:681.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:681.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692347681747 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:47:681.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:47:682.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:683.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:47:683.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692347682925 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:683.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:683.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:683.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692347682979 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:47:683.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:47:729.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:729.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692347729061 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:729.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:729.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:729.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692347729721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:729.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:730.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:47:730.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:47:730.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:730.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692347730099 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:730.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:424.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:424.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692348424754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:424.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:425.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:425.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692348425317 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46737 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:425.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:425.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:48:425.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:48:425.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:425.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692348425757 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:425.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:425.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:425.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692348425765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:425.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:426.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:426.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692348426335 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:48:426.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:48:426.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:426.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692348426764 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:48:426.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:48:426.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:426.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692348426784 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:48:426.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:48:427.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:427.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692348427429 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:427.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:427.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:427.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:48:428.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692348427486 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:48:428.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:48:474.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:474.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692348474638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:474.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:475.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:475.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692348475258 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46083 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:475.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:475.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:48:475.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:48:475.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:475.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692348475679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:475.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:686.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:686.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692348686229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:686.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:686.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:686.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692348686666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35259 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:686.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:686.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:48:686.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:48:686.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:687.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692348686951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:687.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:774.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:774.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692348774318 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:774.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:774.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:774.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692348774505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42493 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:774.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:774.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:48:774.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:48:775.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:775.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692348775016 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:775.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:49:429.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:49:430.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692349429787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:49:430.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:49:430.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:49:430.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692349429983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40629 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:49:430.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:49:430.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:49:430.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:49:430.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:49:430.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692349430438 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:49:430.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:50:701.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:50:701.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692350701273 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:50:701.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:50:701.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:50:701.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692350701462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34281 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:50:701.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:50:701.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:50:701.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:50:701.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:50:701.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692350701883 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:50:701.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:152.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:152.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692351152052 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:152.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:152.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:152.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692351152652 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45849 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:152.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:152.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:51:153.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:51:153.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:153.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692351153078 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:153.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:153.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:153.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692351153084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:153.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:153.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:153.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692351153211 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:51:153.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:51:153.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:153.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692351153316 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:51:153.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:51:153.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:153.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692351153334 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:51:153.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:51:154.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:154.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692351154298 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:154.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:154.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:154.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:51:154.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692351154348 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:51:154.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:51:205.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:205.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692351205512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:205.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:206.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:206.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692351206118 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37337 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:206.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:206.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:51:206.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:51:206.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:206.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692351206411 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:206.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:374.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:374.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692351374556 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:374.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:374.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:375.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692351374976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42661 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:375.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:375.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:51:375.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:51:375.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:375.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692351375461 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:375.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:268.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:268.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692352268559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:268.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:268.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:52:269.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:269.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692352268738 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46347 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:269.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:269.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:52:269.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:269.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692352269294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:269.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:935.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:935.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692352935029 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:935.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:935.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:935.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692352935494 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42825 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:935.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:935.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:52:935.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:52:935.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:935.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692352935805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:935.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:935.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:935.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692352935823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:935.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:936.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:936.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692352936026 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:52:936.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:52:936.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:936.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692352936088 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:52:936.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:52:936.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:936.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692352936615 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:936.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:936.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:936.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692352936641 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:52:936.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:52:936.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:52:987.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:987.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692352987816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:987.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:988.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:988.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692352988374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34407 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:988.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:988.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:52:988.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:52:988.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:988.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692352988796 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:988.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:041.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:042.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692354041911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:042.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:042.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:042.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692354042087 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35083 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:042.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:042.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:54:042.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:54:042.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:042.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692354042512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:042.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:231.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:231.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692354231462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:231.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:232.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:232.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692354232039 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33821 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:232.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:232.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:54:232.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:54:232.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:232.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692354232356 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:232.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:590.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:590.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:54:592.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692354590031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:592.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:592.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:592.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692354590653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37497 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:592.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:592.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:54:592.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:592.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692354592220 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:592.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:592.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:592.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692354592227 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:592.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:592.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:592.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692354592286 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:54:592.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:54:592.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:592.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692354592333 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:54:592.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:54:592.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:592.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692354592909 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:592.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:592.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:593.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692354592973 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:54:593.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:54:592.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:54:644.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:644.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692354644164 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:644.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:644.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:644.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692354644509 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40163 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:644.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:644.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:54:644.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:54:644.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:645.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692354644894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:645.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:083.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:083.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692355083659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:083.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:084.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:084.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692355084275 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36617 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:084.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:084.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:55:084.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:55:084.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:084.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692355084919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:084.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:084.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:085.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692355084925 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:085.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:085.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:085.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692355085008 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:55:085.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:55:085.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:085.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692355085099 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:55:085.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:55:085.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:085.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692355085121 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:55:085.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:55:085.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:085.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692355085732 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:085.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:085.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:085.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692355085773 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:55:085.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:55:085.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:55:133.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:134.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692355133960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:134.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:134.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:134.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692355134158 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40007 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:134.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:134.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:55:134.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:55:134.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:134.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692355134571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:134.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:590.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:590.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692355590064 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:590.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:590.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:590.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692355590699 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39979 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:590.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:590.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:55:590.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:55:591.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:591.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692355591018 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:591.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:56:112.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:56:112.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692356112661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:56:112.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:56:113.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:56:113.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692356113114 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36041 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:56:113.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:56:113.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:56:113.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:56:113.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:56:113.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692356113665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:56:113.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:56:205.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:56:205.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692356205749 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:56:205.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:56:206.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:56:206.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692356206350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44813 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:56:206.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:56:206.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:56:206.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:56:206.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:56:206.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692356206781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:56:206.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:57:571.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:57:571.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692357571150 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:57:571.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:57:571.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:57:571.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692357571743 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40547 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:57:571.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:57:571.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:57:571.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:57:572.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:57:572.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692357572025 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:57:572.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:192.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:192.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692358192556 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:192.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:192.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:192.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692358192774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45451 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:192.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:192.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:58:193.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:58:193.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:193.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692358193073 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:193.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:193.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:193.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692358193078 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:193.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:193.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:193.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692358193154 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:58:193.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:58:193.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:193.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692358193416 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:58:193.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:58:193.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:193.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692358193434 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:58:193.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:58:194.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:194.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692358194003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:194.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:194.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:194.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:58:194.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692358194043 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:58:194.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:58:240.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:240.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692358240263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:240.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:240.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:240.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692358240439 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42973 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:240.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:240.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:58:240.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:58:240.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:240.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692358240851 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:240.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:240.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:240.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692358240890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:241.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:241.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:241.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692358240948 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37119 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:241.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:241.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:58:241.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:58:241.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:241.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692358241351 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:241.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:312.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:312.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692359312005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:312.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:312.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:312.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692359312510 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36195 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:312.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:312.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:59:312.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:59:313.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:313.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692359312982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:313.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:965.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:965.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692359965876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:965.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:966.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:966.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692359966371 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39931 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:966.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:966.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:59:966.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:59:966.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:966.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692359966755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:966.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:966.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:966.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692359966763 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:966.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:966.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:966.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692359966855 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:59:966.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:59:966.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:966.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692359966905 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:59:966.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:59:967.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:967.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692359967448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:967.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:967.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:967.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:59:967.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692359967473 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:59:967.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:00:020.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:00:020.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692360020674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:00:020.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:00:021.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:00:021.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692360020886 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39953 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:00:021.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:00:021.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:00:021.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:00:021.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:00:021.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692360021370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:00:021.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:011.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:011.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692361011596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:011.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:011.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:011.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692361011782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33135 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:011.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:012.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:01:012.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:01:012.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:012.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692361012239 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:012.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:149.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:149.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692361149158 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:149.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:149.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:149.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692361149341 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38389 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:149.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:149.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:01:149.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:01:149.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:149.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692361149809 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:149.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:784.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:784.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692361784211 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:784.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:784.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:784.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692361784633 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35297 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:784.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:784.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:01:785.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:01:785.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:785.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692361785229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:785.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:785.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:785.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692361785272 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:785.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:785.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:785.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692361785321 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:01:785.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:01:785.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:785.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692361785385 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:01:785.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:01:785.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:785.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692361785934 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:785.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:01:785.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:786.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:786.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692361785961 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:01:786.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:01:830.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:830.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692361830395 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 2537 dequeues = 2537 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 2574 dequeues = 2574 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692332849494 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692361786112 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:06:01:830.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:01:830.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:830.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692361830544 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 508 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 52 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 575 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 137 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1946 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 556 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1271, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2537, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:01:830.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:01:832.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:832.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692361832097 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:832.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:832.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:832.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692361832319 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44661 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:832.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:832.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:01:833.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:01:833.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:833.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692361833079 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:833.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:838.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:838.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692361838751 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.25, ] ] ] 2025-09-01 Mon 02:06:01:838.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:06:01:838.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:838.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692361838794 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1371374304 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.015 five_min_avg = 0.14 fifteen_min_avg = 0.155 ] cpu_share = 1.775 ] ] 2025-09-01 Mon 02:06:01:838.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:01:838.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:838.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692361838800 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.775 used_sys_mem = 2854240503 one_min_cpuload = 0.015 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:06:01:838.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:06:01:838.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:839.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692361838826 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 773 in_bytes = 83020 out_tpkts = 818 out_bytes = 45175 exception_packets = 37 exception_packets_allowed = 26 raw_vhost_stats= [ name = vhost0 in_pkts = 773 in_bytes = 83020 out_pkts = 818 out_bytes = 45175 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 697 in_bytes = 40093 out_pkts = 776 out_bytes = 83146 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:06:01:839.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:01:839.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:839.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692361838850 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:06:01:839.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:06:02:417.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:417.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692362417602 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:417.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:418.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:418.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692362418052 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43251 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:418.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:418.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:02:418.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:02:418.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:418.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692362418557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:418.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:418.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:418.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692362418565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:418.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:418.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:418.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692362418639 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:02:418.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:02:418.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:418.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692362418745 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:02:418.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:02:418.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:418.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692362418767 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:02:418.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:02:419.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:419.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692362419336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:419.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:419.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:419.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:02:419.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692362419378 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:02:419.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:02:465.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:465.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692362465597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:465.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:466.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:466.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692362466222 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45813 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:466.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:466.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:02:466.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:02:466.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:466.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692362466540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:466.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:778.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:778.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692362778262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:778.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:778.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:778.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692362778914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46295 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:778.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:779.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:02:779.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:02:779.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:779.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692362779375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:779.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:849.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:06:02:849.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:06:02:850.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:02:850.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692362849702 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:02:850.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:02:850.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:02:850.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:06:02:850.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:06:02:850.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692362850065 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 512 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 579 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 138 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1960 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 560 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1280, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2562, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:02:850.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:02:850.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:06:02:850.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:06:02:850.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:02:850.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Idle 2025-09-01 Mon 02:06:02:850.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:850.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692362850438 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 513 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 579 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 138 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1960 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 560 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1280, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2564, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:02:850.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:02:850.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:850.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692362850468 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:02:850.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:02:850.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:06:02:850.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:850.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692362850557 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 513 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 580 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 138 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1960 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 560 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1281, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2564, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:02:850.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:02:938.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:938.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692362938400 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:938.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:938.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:938.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692362938855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35063 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:938.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:939.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:02:939.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:02:939.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:939.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692362939404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:939.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:03:483.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:03:483.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692363483710 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:03:484.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:03:484.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:03:484.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692363483911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44777 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:03:484.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:03:484.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:03:484.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:03:484.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:03:484.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692363484661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:03:484.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:04:813.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:04:813.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692364813568 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:04:813.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:04:813.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:04:813.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692364813743 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38937 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:04:813.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:04:814.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:04:814.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:04:814.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:04:814.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692364814363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:04:814.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:290.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:290.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692365290024 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:290.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:290.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:290.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692365290285 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41173 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:290.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:290.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:05:290.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:05:290.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:290.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692365290830 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:290.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:290.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:290.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692365290845 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:290.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:290.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:290.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692365290901 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:05:290.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:05:290.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:291.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692365290983 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:05:291.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:05:291.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:291.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692365291002 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:05:291.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:05:291.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:291.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692365291546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:291.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:291.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:291.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692365291581 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:05:291.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:05:291.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:05:342.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:342.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692365342304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:342.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:342.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:342.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692365342667 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38263 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:342.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:342.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:05:343.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:05:343.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:343.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692365343103 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:343.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:608.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:609.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692365608801 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:609.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:609.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:609.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692365609004 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38703 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:609.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:609.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:05:609.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:05:609.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:609.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692365609383 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:609.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:360.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:360.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692366360249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:360.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:360.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:360.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692366360775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39507 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:360.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:361.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:06:361.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:06:361.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:361.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692366361748 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:361.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:611.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:611.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692366611548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:611.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:611.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:611.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692366611733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46559 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:611.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:612.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:06:612.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:06:612.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:612.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692366612542 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:612.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:612.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:612.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692366612557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:612.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:612.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:612.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692366612614 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:06:612.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:06:612.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:612.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692366612663 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:06:612.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:06:613.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:613.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692366613189 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:613.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:613.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:613.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692366613237 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:06:613.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:06:613.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:06:666.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:666.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692366666614 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:666.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:667.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:667.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692366667194 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40527 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:667.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:667.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:06:667.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:06:667.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:667.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692366667487 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:667.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:07:729.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:729.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692367729657 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:07:729.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:07:730.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:730.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692367730149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46643 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:07:730.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:07:730.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:07:730.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:07:730.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:730.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692367730497 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:07:730.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:377.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:378.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692368377891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:378.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:378.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:378.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692368378066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41869 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:378.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:378.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:08:378.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:08:378.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:378.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692368378439 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:378.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:486.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:486.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692368486571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:486.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:487.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:487.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692368487209 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46833 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:487.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:487.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:08:487.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:08:487.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:487.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692368487705 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:487.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:487.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:487.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692368487713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:487.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:487.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:487.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692368487769 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:08:487.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:08:487.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:487.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692368487842 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:08:487.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:08:488.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:488.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692368488420 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:488.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:488.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:488.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692368488462 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:08:488.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:08:488.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:08:535.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:535.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692368535737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:535.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:536.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:536.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692368536119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:536.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:536.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:08:536.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:08:536.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:536.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692368536561 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:536.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:490.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:490.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692369490707 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:490.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:490.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:490.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692369490883 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33351 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:490.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:491.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:09:491.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:09:491.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:491.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692369491295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:491.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:497.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:497.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692369497530 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:497.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:497.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:497.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692369497642 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43561 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:497.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:497.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:09:498.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:09:498.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:498.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692369498196 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:498.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:498.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:498.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692369498202 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:498.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:498.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:498.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692369498281 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:09:498.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:09:498.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:498.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692369498374 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:09:498.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:09:498.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:498.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692369498392 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:09:498.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:09:499.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:499.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692369498975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:499.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:499.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:499.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692369499041 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:09:499.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:09:499.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:09:549.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:549.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692369549216 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:549.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:549.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:549.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692369549438 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37057 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:549.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:549.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:09:549.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:09:549.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:550.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692369549953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:550.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:800.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:800.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692369800673 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:800.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:800.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:800.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692369800854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41477 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:800.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:801.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:09:801.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:09:801.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:801.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692369801175 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:801.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:10:612.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:10:612.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692370612094 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:10:612.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:10:612.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:10:612.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692370612284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36213 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:10:612.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:10:612.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:10:612.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:10:612.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:10:612.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692370612809 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:10:612.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:471.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:471.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692371471484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:471.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:472.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:472.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692371472115 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40057 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:472.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:472.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:11:472.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:11:472.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:472.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692371472609 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:472.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:338.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:338.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692372338629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:338.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:338.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:338.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692372338837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44547 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:338.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:339.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:12:339.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:12:339.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:339.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692372339230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:339.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:339.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:339.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692372339238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:339.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:339.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:339.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692372339297 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:12:339.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:12:339.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:339.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692372339409 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:12:339.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:12:339.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:340.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692372339427 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:12:340.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:12:340.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:340.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692372339971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:340.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:340.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:340.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692372340005 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:12:340.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:12:340.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:12:385.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:385.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692372385156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:385.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:385.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:385.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692372385355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40945 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:385.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:385.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:12:385.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:12:385.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:385.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692372385732 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:385.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:755.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:755.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692372754998 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:755.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:755.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:755.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692372755227 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39545 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:755.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:755.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:12:755.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:12:755.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:755.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692372755697 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:755.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:438.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:439.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:13:439.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:13:439.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692373438909 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:439.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:439.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:439.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692373439088 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42777 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:439.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:439.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:439.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692373439421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:439.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:473.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:473.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692373473307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:473.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:473.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:473.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692373473931 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37779 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:474.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:474.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:13:474.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:13:474.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:474.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692373474219 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:474.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:474.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:474.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692373474226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:474.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:474.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:474.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692373474303 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:13:474.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:13:474.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:474.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692373474357 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:13:474.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:13:474.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:474.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692373474875 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:474.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:474.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:474.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692373474932 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:13:475.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:13:474.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:13:522.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:522.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692373522046 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:522.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:522.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:522.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692373522655 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35175 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:522.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:522.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:13:523.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:13:523.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:523.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692373523068 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:523.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:14:459.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:14:459.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692374459225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:14:459.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:14:459.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:14:459.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692374459704 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41411 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:14:459.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:14:460.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:14:460.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:14:460.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:14:460.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692374460207 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:14:460.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:419.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:419.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692375419565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:419.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:420.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:420.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692375420022 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40069 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:420.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:420.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:15:420.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:15:420.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:420.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692375420359 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:420.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:612.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:612.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692375612776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:613.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:613.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:613.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692375613008 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37373 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:613.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:613.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:15:613.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:15:613.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:613.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692375613447 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:613.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:613.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:613.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692375613455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:613.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:613.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:613.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692375613675 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:15:613.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:15:613.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:613.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692375613734 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:15:613.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:15:614.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:614.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:15:614.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692375614293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:614.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:614.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:614.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692375614335 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:15:614.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:15:660.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:660.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692375660486 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:660.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:661.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:661.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692375661082 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33197 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:661.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:661.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:15:661.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:15:661.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:661.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692375661387 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:661.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:941.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:59248 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:15:950.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:59250 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:16:404.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:404.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692376404383 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:404.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:404.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:404.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692376404564 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34539 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:404.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:404.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:16:404.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:16:405.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:405.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692376404994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:405.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:463.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:463.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692376463847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:463.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:464.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:464.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692376464405 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46763 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:464.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:464.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:16:464.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:16:464.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:464.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692376464824 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:464.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:464.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:464.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692376464831 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:464.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:464.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:464.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692376464909 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:16:464.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:16:465.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:465.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692376465017 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:16:465.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:16:465.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:465.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692376465037 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:16:465.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:16:465.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:465.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692376465717 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:465.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:16:465.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:465.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:465.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692376465754 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:16:465.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:16:510.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:510.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692376510870 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:510.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:511.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:511.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692376511305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43529 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:511.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:511.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:16:512.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:16:512.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:512.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692376512151 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:512.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:696.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:696.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692376696521 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:696.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:696.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:696.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692376696710 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40485 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:696.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:696.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:16:696.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:16:697.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:697.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692376697028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:697.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:17:556.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:17:556.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692377556327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:17:556.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:17:556.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:17:556.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692377556706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34079 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:17:556.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:17:557.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:17:557.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:17:557.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:17:557.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692377557348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:17:557.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:749.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:749.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692378749227 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:749.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:749.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:749.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692378749676 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43935 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:749.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:749.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:18:749.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:18:749.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:750.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692378749958 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:750.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:164.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:164.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692379164532 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:164.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:164.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:164.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692379164722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33669 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:164.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:164.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:19:164.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:19:165.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:165.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692379165009 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:165.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:165.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:165.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692379165014 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:165.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:165.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:165.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692379165103 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:19:165.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:19:165.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:165.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692379165205 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:19:165.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:19:165.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:165.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692379165222 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:19:165.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:19:165.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:165.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692379165772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:165.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:165.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:165.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:19:165.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692379165808 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:19:165.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:19:212.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:212.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692379211980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:212.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:212.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:212.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692379212481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41367 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:212.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:212.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:19:212.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:19:213.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:213.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692379213022 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:213.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:537.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:537.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692379537481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:537.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:537.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:537.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692379537740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39983 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:537.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:538.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:19:538.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:19:538.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:538.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692379538304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:538.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:140.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:140.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692380140187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:140.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:140.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:140.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692380140666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38225 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:140.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:141.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:20:141.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:20:141.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:141.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692380141215 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:141.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:293.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:293.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692380293102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:293.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:293.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:293.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692380293293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34515 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:293.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:293.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:20:293.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:20:293.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:293.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692380293748 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:294.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:294.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:294.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692380293766 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:294.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:294.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:294.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692380293828 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:20:294.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:20:294.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:294.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692380293896 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:20:294.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:20:294.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:294.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692380294462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:294.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:294.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:294.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692380294535 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:20:294.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:20:294.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:20:343.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:343.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692380343643 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:343.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:343.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:343.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692380343834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45601 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:343.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:344.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:20:344.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:20:344.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:344.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692380344224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:344.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:334.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:334.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692381334387 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:334.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:334.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:334.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692381334869 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33629 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:334.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:335.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:21:335.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:21:335.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:335.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692381335297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:335.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:247.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:247.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692382247412 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:247.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:247.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:247.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692382247911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40473 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:247.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:248.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:22:248.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:22:248.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:248.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692382248425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:248.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:854.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:854.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692382854141 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:854.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:854.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:854.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692382854667 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36939 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:854.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:854.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:22:855.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:22:855.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:855.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692382855073 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:855.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:855.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:855.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692382855081 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:855.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:855.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:855.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692382855206 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:22:855.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:22:855.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:855.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692382855254 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:22:855.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:22:855.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:855.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692382855775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:855.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:855.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:855.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:22:855.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692382855823 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:22:855.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:22:901.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:902.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692382901918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:902.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:902.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:902.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692382902357 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41721 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:902.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:902.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:22:902.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:22:902.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:902.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692382902826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:902.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:423.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:423.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692383423426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:423.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:424.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:424.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692383424037 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34337 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:424.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:424.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:23:424.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:23:424.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:424.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692383424620 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:424.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:786.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:786.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692383786446 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:786.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:786.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:786.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692383786959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43447 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:786.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:787.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:23:787.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:23:787.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:787.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692383787432 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:787.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:787.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:787.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692383787440 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:787.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:787.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:787.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692383787535 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:23:787.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:23:787.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:787.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692383787637 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:23:787.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:23:787.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:787.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692383787655 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:23:787.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:23:788.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:788.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692383788187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:788.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:788.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:788.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692383788215 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:23:788.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:23:788.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:23:830.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:830.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692383830053 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:830.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:830.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:830.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692383830240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44167 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:830.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:830.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:23:830.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:23:830.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:830.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692383830674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:830.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:833.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:833.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692383833313 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:833.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:833.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:833.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:23:833.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692383833453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33233 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:833.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:833.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:23:833.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:833.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692383833792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:833.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:24:796.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:24:797.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692384796947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:24:797.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:24:797.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:24:797.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:24:797.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:24:797.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692384797144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37975 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:24:797.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:24:797.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:24:797.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692384797507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:24:797.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:792.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:792.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692385792822 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:792.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:793.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:793.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692385793481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34351 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:793.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:793.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:25:793.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:25:793.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:793.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692385793921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:793.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:884.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:884.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692385884569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:884.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:885.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:885.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692385885173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44979 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:885.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:885.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:25:885.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:25:885.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:885.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692385885754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:885.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:885.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:885.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692385885759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:885.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:885.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:885.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692385885834 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:25:885.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:25:885.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:886.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692385885921 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:25:886.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:25:886.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:886.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692385885939 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:25:886.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:25:886.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:886.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692385886510 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:886.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:886.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:886.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:25:886.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692385886538 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:25:886.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:25:936.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:936.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692385936691 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:936.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:937.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:937.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692385937435 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34515 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:937.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:937.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:25:937.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:25:937.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:937.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692385937840 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:937.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:26:777.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:26:777.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692386777669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:26:777.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:26:778.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:26:778.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692386778146 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34261 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:26:778.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:26:778.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:26:778.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:26:778.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:26:778.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692386778568 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:26:778.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:26:901.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:26:901.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692386900971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:26:901.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:26:901.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:26:901.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692386901421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41903 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:26:901.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:26:901.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:26:901.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:26:901.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:26:901.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692386901898 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:26:901.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:456.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:456.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692387456816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:456.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:457.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:457.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692387457274 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45429 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:457.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:457.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:27:457.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:27:457.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:458.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692387457714 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:458.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:458.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:458.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692387457720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:458.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:458.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:458.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692387457779 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:27:458.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:27:458.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:458.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692387457831 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:27:458.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:27:458.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:458.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692387458370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:458.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:458.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:458.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:27:458.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692387458426 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:27:458.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:27:503.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:503.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692387503528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:503.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:503.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:503.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692387503701 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37869 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:503.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:503.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:27:503.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:27:504.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:504.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692387504034 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:504.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:467.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:467.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692388467209 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:467.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:467.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:467.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692388467731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37485 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:467.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:468.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:28:468.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:28:468.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:468.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692388468224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:468.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:846.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:846.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692388846100 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:846.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:846.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:846.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692388846309 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42965 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:846.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:846.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:28:846.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:28:846.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:846.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692388846732 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:846.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:646.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:646.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692389646057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:646.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:646.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:646.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692389646587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42721 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:646.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:646.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:29:647.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:29:647.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:647.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692389647105 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:647.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:647.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:647.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692389647112 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:647.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:647.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:647.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692389647192 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:29:647.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:29:647.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:647.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692389647242 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:29:647.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:29:647.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:647.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692389647759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:647.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:647.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:647.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:29:647.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692389647828 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:29:647.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:29:692.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:693.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692389692943 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:693.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:693.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:693.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692389693634 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39397 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:693.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:693.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:29:694.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:29:694.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:694.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692389694348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:694.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:502.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:502.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692390502393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:502.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:502.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:30:502.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:30:503.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:503.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692390502579 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46391 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:503.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:503.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:503.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692390502946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:503.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:522.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:522.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692390522718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:523.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:523.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:30:523.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:30:523.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:523.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692390522878 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41975 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:523.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:523.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:523.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692390523342 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:523.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:523.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:523.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692390523346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:523.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:523.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:523.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692390523448 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:30:523.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:30:523.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:523.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692390523636 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:30:523.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:30:523.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:523.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692390523661 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:30:523.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:30:524.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:524.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:30:524.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692390524241 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:524.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:524.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:524.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692390524282 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:30:524.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:30:576.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:576.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692390576432 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:576.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:576.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:576.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692390576881 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45199 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:576.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:577.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:30:577.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:30:577.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:577.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692390577351 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:577.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:729.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:729.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692390729530 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:729.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:730.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:730.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692390730173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38595 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:730.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:730.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:30:730.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:30:730.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:730.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692390730537 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:730.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:31:531.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:31:531.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692391531505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:31:531.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:31:532.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:31:532.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692391532162 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36349 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:31:532.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:31:532.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:31:532.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:31:532.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:31:532.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692391532549 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:31:532.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:31:830.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:31:830.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692391830727 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 2853 dequeues = 2853 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 2893 dequeues = 2893 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692362850521 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692391532557 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:06:31:831.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:31:831.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:31:831.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692391830833 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 570 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 57 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2190 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 626 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1431, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2853, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:31:831.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:31:839.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:31:839.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692391839025 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:06:31:839.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:06:31:839.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:31:839.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692391839069 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 873 in_bytes = 100162 out_tpkts = 921 out_bytes = 51137 exception_packets = 40 exception_packets_dropped = 12 exception_packets_allowed = 28 raw_vhost_stats= [ name = vhost0 in_pkts = 873 in_bytes = 100162 out_pkts = 921 out_bytes = 51137 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 6.77084e-15 total_out_bandwidth_utilization = 1.66913e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1249, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 3079, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 785 in_bytes = 45425 out_pkts = 876 out_bytes = 100288 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:06:31:839.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:31:839.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:31:839.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692391839094 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:06:31:839.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:06:32:674.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:674.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692392674699 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:674.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:674.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:674.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692392674888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33087 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:674.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:675.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:32:675.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:32:675.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:675.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692392675332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:675.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:850.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:06:32:850.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:06:32:851.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:32:851.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692392850762 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:32:851.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:32:851.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:32:851.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:06:32:851.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:06:32:851.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692392851037 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 570 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 59 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 649 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2193 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 627 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1433, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2861, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:32:851.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:32:851.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:06:32:851.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:06:32:851.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:32:851.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Idle 2025-09-01 Mon 02:06:32:851.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:851.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692392851426 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 571 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 649 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2193 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 627 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1433, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2863, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:32:851.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:32:851.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:851.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692392851450 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:32:851.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:32:851.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:06:32:851.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:851.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692392851542 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 571 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 650 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2193 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 627 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1434, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2863, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:32:851.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:33:058.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:059.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692393058916 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:059.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:059.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:059.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692393059450 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42193 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:059.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:059.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:33:059.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:33:059.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:059.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692393059911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:059.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:059.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:059.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692393059918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:059.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:060.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:060.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692393060059 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:33:060.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:33:060.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:060.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692393060195 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:33:060.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:33:060.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:060.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692393060220 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:33:060.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:33:060.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:060.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692393060795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:060.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:060.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:060.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692393060849 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:33:060.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:33:060.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:33:108.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:109.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692393108915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:109.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:109.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:109.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692393109343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38489 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:109.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:109.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:33:109.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:33:109.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:109.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692393109703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:109.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:350.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:350.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692393350685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:350.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:351.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:351.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692393351289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44115 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:351.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:351.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:33:351.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:33:351.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:351.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692393351765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:351.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:072.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:072.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692394072890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:072.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:073.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:073.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692394073353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39049 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:073.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:073.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:34:073.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:34:073.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:073.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692394073659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:073.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:571.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:571.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692394571107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:571.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:571.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:571.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692394571646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41481 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:571.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:571.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:34:571.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:34:572.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:572.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692394572019 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:572.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:572.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:572.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692394572030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:572.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:572.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:572.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692394572115 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:34:572.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:34:572.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:572.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692394572177 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:34:572.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:34:572.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:572.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692394572720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:572.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:572.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:572.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:34:572.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692394572774 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:34:572.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:34:625.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:626.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692394625929 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:626.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:626.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:626.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692394626506 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41637 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:626.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:626.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:34:626.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:34:626.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:627.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692394626984 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:627.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:35:572.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:35:572.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692395572124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:35:572.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:35:572.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:35:572.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692395572659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40777 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:35:572.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:35:573.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:35:573.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:35:573.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:35:573.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692395573170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:35:573.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:36:161.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:161.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692396161815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:36:161.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:36:162.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:162.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692396162280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39379 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:36:162.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:36:162.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:36:162.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:36:162.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:162.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692396162600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:36:162.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:36:851.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:851.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692396851482 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:36:851.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:36:852.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:852.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692396852005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45179 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:36:852.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:36:852.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:36:852.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:36:852.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:852.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692396852552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:36:852.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:36:852.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:852.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692396852562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:36:852.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:36:852.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:852.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692396852620 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:36:852.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:36:852.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:852.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692396852666 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:36:852.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:36:853.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:853.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692396853256 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:36:853.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:36:853.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:853.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:36:853.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692396853307 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:36:853.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:36:905.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:905.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692396905458 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:36:905.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:36:905.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:905.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692396905683 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42907 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:36:905.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:36:905.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:36:905.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:36:906.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:36:906.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692396906008 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:36:906.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:239.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:240.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692397239899 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:240.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:240.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:240.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692397240264 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42653 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:240.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:240.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:37:240.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:37:240.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:240.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692397240788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:240.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:240.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:241.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692397240795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:241.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:241.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:241.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692397240862 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:37:241.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:37:241.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:241.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692397241315 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:37:241.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:37:241.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:241.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692397241334 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:37:241.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:37:241.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:241.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692397241893 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:241.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:241.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:241.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692397241920 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:37:241.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:37:241.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:37:287.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:287.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692397287099 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:287.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:287.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:287.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692397287598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37339 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:287.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:287.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:37:288.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:37:288.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:288.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692397288377 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:288.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:607.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:607.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692397607312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:607.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:607.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:607.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692397607762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43453 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:607.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:608.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:37:608.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:37:608.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:608.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692397608416 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:608.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:905.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:905.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692397905101 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:905.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:905.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:905.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:37:905.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:37:905.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692397905289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35027 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:905.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:37:905.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:37:905.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692397905671 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:37:905.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:38:224.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:38:224.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692398224551 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:38:224.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:38:225.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:38:225.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692398225196 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35593 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:38:225.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:38:225.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:38:225.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:38:225.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:38:225.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692398225608 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:38:225.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:723.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:723.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692399723776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:723.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:723.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:724.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692399723990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39499 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:724.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:724.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:39:724.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:39:724.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:724.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692399724468 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:724.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:834.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:834.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692399834772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:834.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:835.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:835.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692399835291 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42751 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:835.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:835.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:39:835.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:39:835.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:835.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692399835647 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:835.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:835.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:835.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692399835654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:835.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:835.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:835.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692399835727 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:39:835.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:39:835.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:835.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692399835814 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:39:835.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:39:835.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:835.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692399835832 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:39:835.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:39:836.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:39:836.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:836.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692399836385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:836.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:836.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:836.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692399836430 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:39:836.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:39:881.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:881.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692399881567 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:881.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:881.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:881.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692399881736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43639 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:881.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:882.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:39:882.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:39:882.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:882.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692399882181 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:882.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:40:259.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:40:259.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692400259760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:40:259.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:40:260.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:40:260.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692400260234 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42753 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:40:260.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:40:260.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:40:260.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:40:260.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:40:260.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692400260737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:40:260.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:40:944.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:40:944.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692400944349 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:40:944.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:40:944.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:40:945.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692400944950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45271 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:40:945.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:40:945.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:40:945.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:40:945.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:40:945.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692400945394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:40:945.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:41:748.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:41:748.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692401748543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:41:748.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:41:749.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:41:749.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692401749731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36429 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:41:749.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:41:750.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:41:750.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:41:750.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:41:750.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692401750314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:41:750.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:41:750.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:41:750.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692401750319 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:41:750.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:41:751.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:41:751.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692401750456 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:41:751.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:41:751.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:41:751.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692401750524 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:41:751.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:41:751.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:41:751.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692401751031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:41:751.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:41:751.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:41:751.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692401751057 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:41:751.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:41:751.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:41:800.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:41:800.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692401800166 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:41:800.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:41:800.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:41:800.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692401800544 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34111 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:41:800.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:41:800.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:41:800.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:41:800.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:41:800.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692401800928 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:41:800.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:42:746.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:42:746.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692402746053 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:42:746.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:42:746.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:42:746.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692402746530 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35327 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:42:746.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:42:746.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:42:746.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:42:747.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:42:747.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692402746999 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:42:747.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:42:799.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:42:799.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692402799527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:42:799.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:42:799.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:42:799.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692402799973 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42367 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:42:800.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:42:800.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:42:800.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:42:800.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:42:800.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692402800452 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:42:800.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:43:756.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:756.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692403756629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:43:756.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:43:757.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:757.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692403757099 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36589 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:43:757.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:43:757.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:43:757.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:43:757.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:757.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692403757486 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:43:757.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:43:757.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:757.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692403757494 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:43:757.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:43:757.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:757.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692403757572 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:43:757.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:43:757.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:757.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692403757779 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:43:757.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:43:758.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:758.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692403758317 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:43:758.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:43:758.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:758.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:43:758.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692403758357 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:43:758.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:43:806.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:806.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692403806455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:43:806.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:43:806.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:806.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692403806871 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36929 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:43:806.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:43:807.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:43:807.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:43:807.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:807.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692403807381 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:43:807.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:508.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:509.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692404508878 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:509.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:509.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:509.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692404509077 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41655 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:509.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:509.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:44:509.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:44:509.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:509.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692404509544 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:509.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:509.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:509.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692404509559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:509.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:509.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:509.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692404509618 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:44:509.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:44:509.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:509.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692404509872 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:44:509.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:44:509.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:509.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692404509891 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:44:510.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:44:510.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:510.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692404510508 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:510.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:510.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:510.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:44:510.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692404510538 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:44:510.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:44:562.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:562.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692404562724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:562.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:562.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:562.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692404562903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42433 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:563.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:563.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:44:563.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:44:563.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:563.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692404563255 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:563.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:799.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:799.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692404799205 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:799.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:799.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:799.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692404799765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45839 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:799.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:800.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:44:800.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:44:800.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:800.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692404800180 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:800.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:824.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:824.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692404824503 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:824.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:824.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:824.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692404824677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43891 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:824.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:44:824.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:44:825.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:44:825.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:825.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692404825080 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:44:825.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:45:535.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:45:535.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:45:535.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692405535325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:45:535.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:45:535.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:45:535.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692405535500 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40959 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:45:535.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:45:535.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:45:535.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:45:535.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692405535855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:45:535.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:688.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:688.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692406688663 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:46:688.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:688.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:688.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692406688842 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38301 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:46:688.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:689.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:46:689.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:46:689.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:689.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692406689284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:46:689.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:689.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:689.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692406689290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:46:689.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:689.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:689.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692406689367 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:46:689.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:46:689.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:689.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692406689454 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:46:689.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:46:689.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:689.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692406689472 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:46:689.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:46:690.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:690.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692406690247 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:46:690.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:690.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:690.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:46:690.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692406690302 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:46:690.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:46:740.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:740.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692406740436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:46:740.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:740.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:740.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692406740852 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42057 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:46:740.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:741.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:46:741.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:46:741.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:741.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692406741468 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:46:741.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:841.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:841.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692406841222 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:46:841.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:841.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:841.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692406841793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46719 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:46:841.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:842.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:46:842.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:46:842.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:46:842.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692406842205 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:46:842.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:47:462.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:47:462.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692407462029 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:47:462.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:47:462.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:47:462.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692407462610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46813 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:47:462.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:47:462.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:47:462.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:47:462.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:47:462.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692407462906 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:47:462.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:47:803.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:47:803.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692407803656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:47:803.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:47:804.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:47:804.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692407804147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45565 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:47:804.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:47:804.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:47:804.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:47:804.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:47:804.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692407804585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:47:804.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:48:796.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:48:796.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692408796382 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:48:796.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:48:796.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:48:797.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692408796947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36713 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:48:797.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:48:797.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:48:797.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:48:797.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:48:797.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692408797323 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:48:797.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:48:797.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:48:797.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692408797331 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:48:797.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:48:797.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:48:797.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692408797411 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:48:797.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:48:797.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:48:797.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692408797462 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:48:797.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:48:798.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:48:798.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692408797993 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:48:798.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:48:798.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:48:798.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:48:798.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692408798036 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:48:798.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:48:851.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:48:851.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:48:851.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:48:851.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692408851225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:48:851.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:48:851.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:48:851.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692408851406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40977 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:48:851.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:48:851.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:48:851.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692408851735 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:48:851.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:49:796.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:49:796.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692409796892 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:49:797.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:49:797.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:49:797.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692409797085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33217 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:49:797.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:49:797.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:49:797.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:49:797.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:49:797.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692409797420 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:49:797.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:49:892.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:49:892.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692409892748 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:49:892.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:49:893.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:49:893.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692409892932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39977 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:49:893.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:49:893.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:49:893.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:49:893.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:49:893.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692409893326 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:49:893.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:50:622.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:622.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692410622390 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:622.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:50:622.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:622.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692410622605 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46581 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:622.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:50:622.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:50:623.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:50:623.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:623.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692410623108 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:623.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:50:623.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:623.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692410623121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:623.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:50:623.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:623.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:50:623.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692410623198 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:50:624.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:50:624.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:624.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692410623247 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:50:624.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:50:624.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:624.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692410623755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:624.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:50:624.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:624.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692410623791 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:50:624.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:50:669.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:669.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692410668987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:669.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:50:669.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:669.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692410669183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38501 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:669.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:50:669.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:50:669.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:50:669.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:669.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692410669621 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:669.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:641.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:641.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692411641745 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:641.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:642.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:642.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692411642363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38797 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:642.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:642.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:51:642.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:51:642.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:642.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692411642753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:642.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:747.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:747.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692411747040 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:747.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:747.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:747.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692411747703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42167 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:747.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:747.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:51:747.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:51:748.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:748.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692411748029 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:748.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:748.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:748.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692411748047 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:748.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:748.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:748.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692411748107 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:51:748.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:51:748.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:748.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692411748363 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:51:748.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:51:748.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:748.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692411748383 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:51:748.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:51:748.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:748.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692411748904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:748.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:748.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:748.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:51:748.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692411748947 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:51:749.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:51:797.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:797.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692411797106 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:797.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:797.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:797.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692411797667 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39897 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:797.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:797.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:51:798.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:51:798.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:798.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692411798065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:798.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:813.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:813.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692411813504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:813.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:813.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:813.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692411813685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39437 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:813.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:51:813.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:51:813.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:51:814.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:51:814.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692411814019 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:51:814.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:52:806.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:52:806.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692412806229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:52:806.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:52:806.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:52:806.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692412806428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45167 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:52:806.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:52:806.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:52:806.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:52:806.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:52:806.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692412806892 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:52:806.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:53:568.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:569.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692413568911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:53:569.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:53:569.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:569.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692413569589 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33071 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:53:569.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:53:569.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:53:569.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:53:570.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:570.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692413570007 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:53:570.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:53:637.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:637.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692413637456 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:53:637.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:53:638.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:638.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692413638032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34509 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:53:638.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:53:638.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:53:638.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:53:638.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:638.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692413638553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:53:638.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:53:638.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:638.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692413638559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:53:638.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:53:638.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:638.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692413638636 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:53:638.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:53:638.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:638.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692413638745 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:53:638.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:53:638.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:638.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692413638769 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:53:638.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:53:639.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:639.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692413639319 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:53:639.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:53:639.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:639.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692413639366 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:53:639.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:53:639.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:53:690.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:690.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692413690542 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:53:690.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:53:691.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:691.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692413691149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37837 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:53:691.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:53:691.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:53:691.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:53:691.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:53:691.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692413691443 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:53:691.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:54:672.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:672.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692414672632 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:54:672.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:54:673.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:673.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692414673151 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45589 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:54:673.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:54:673.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:54:673.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:54:673.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:673.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692414673552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:54:673.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:54:859.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:859.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692414859060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:54:859.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:54:859.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:859.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692414859354 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46363 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:54:859.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:54:859.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:54:859.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:54:859.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:859.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692414859787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:54:859.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:55:594.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:594.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692415594197 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:55:594.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:55:594.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:594.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692415594746 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46869 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:55:594.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:55:594.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:55:595.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:55:595.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:595.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692415595141 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:55:595.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:55:595.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:595.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692415595150 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:55:595.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:55:595.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:595.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692415595209 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:55:595.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:55:595.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:595.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692415595258 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:55:595.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:55:595.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:595.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692415595792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:55:595.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:55:595.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:595.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692415595835 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:55:595.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:55:595.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:55:644.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:644.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692415644092 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:55:644.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:55:644.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:644.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692415644272 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34257 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:55:644.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:55:644.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:55:644.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:55:644.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:644.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692415644775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:55:644.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:56:544.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:56:545.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692416544919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:56:545.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:56:545.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:56:545.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692416545351 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40245 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:56:545.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:56:545.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:56:545.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:56:545.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:56:545.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692416545961 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:56:546.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:56:707.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:56:707.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692416707721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:56:707.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:56:707.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:56:707.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692416707920 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38299 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:56:707.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:56:708.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:56:708.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:56:708.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:56:708.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692416708284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:56:708.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:57:530.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:57:530.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692417530105 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:57:530.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:57:530.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:57:530.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692417530576 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44571 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:57:530.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:57:530.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:57:531.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:57:531.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:57:531.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692417531239 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:57:531.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:57:531.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:57:531.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692417531247 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:57:531.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:57:531.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:57:531.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692417531304 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:57:531.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:57:531.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:57:531.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692417531360 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:57:531.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:57:531.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:57:531.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692417531896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:57:531.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:57:531.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:57:531.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692417531940 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:57:531.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:57:532.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:57:581.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:57:581.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692417581240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:57:581.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:57:581.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:57:581.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692417581834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35351 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:57:581.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:57:582.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:57:582.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:57:582.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:57:582.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692417582137 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:57:582.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:531.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:532.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692418531926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:532.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:532.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:532.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692418532403 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38953 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:532.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:532.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:58:532.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:58:532.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:532.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692418532953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:532.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:533.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:533.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692418532959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:533.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:533.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:533.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692418533017 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:58:533.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:58:533.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:533.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692418533098 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:58:533.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:58:533.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:533.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692418533117 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:58:533.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:58:533.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:533.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692418533793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:533.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:533.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:533.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692418533832 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:58:533.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:58:533.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:58:544.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:544.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692418544065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:544.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:544.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:544.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692418544591 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36899 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:544.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:544.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:58:544.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:58:545.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:545.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692418545000 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:545.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:545.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:545.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692418545311 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:545.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:545.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:545.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692418545406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41493 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:545.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:545.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:58:545.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:58:545.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:545.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692418545688 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:545.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:579.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:580.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692418579944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:580.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:580.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:580.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692418580121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42051 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:580.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:580.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:58:580.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:58:580.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:580.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692418580484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:580.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:59:516.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:59:516.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692419516694 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:59:516.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:59:517.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:59:517.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692419517224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36653 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:59:517.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:59:517.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:59:517.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:59:517.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:59:517.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692419517687 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:59:517.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:00:489.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:489.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692420489834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:00:489.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:00:490.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:490.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692420490441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41657 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:00:490.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:00:490.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:00:490.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:00:490.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:490.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692420490758 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:00:490.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:00:920.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:920.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692420920457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:00:920.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:00:921.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:921.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692420921121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39991 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:00:921.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:00:921.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:00:921.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:00:921.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:921.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692420921459 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:00:921.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:00:921.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:921.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692420921466 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:00:921.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:00:921.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:921.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692420921556 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:00:921.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:00:921.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:921.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692420921869 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:00:921.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:00:921.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:921.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692420921888 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:00:921.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:00:922.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:922.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692420922436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:00:922.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:00:922.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:922.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692420922471 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:00:922.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:00:922.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:00:973.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:973.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692420973750 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:00:973.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:00:974.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:974.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692420974246 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40007 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:00:974.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:00:974.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:00:974.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:00:974.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:00:974.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692420974835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:00:974.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:01:497.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:01:497.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692421497833 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:01:497.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:01:498.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:01:498.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692421498507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40527 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:01:498.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:01:498.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:01:498.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:01:498.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:01:498.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692421498845 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:01:498.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:01:831.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:01:831.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692421830929 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 3161 dequeues = 3161 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 3204 dequeues = 3204 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692392851506 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692421498853 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:07:01:831.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:01:831.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:01:831.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692421831035 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 632 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 717 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 171 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2428 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 694 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1585, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3161, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:01:831.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:01:839.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:01:839.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692421839199 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:07:01:839.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:07:01:839.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:01:839.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692421839249 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1379771104 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.005 five_min_avg = 0.11 fifteen_min_avg = 0.145 ] cpu_share = 1.74167 ] ] 2025-09-01 Mon 02:07:01:839.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:07:01:839.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:01:839.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692421839255 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.74167 used_sys_mem = 2854240503 one_min_cpuload = 0.005 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:07:01:839.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:07:01:839.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:01:839.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692421839282 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 960 in_bytes = 106024 out_tpkts = 1009 out_bytes = 55689 exception_packets = 41 exception_packets_allowed = 29 raw_vhost_stats= [ name = vhost0 in_pkts = 960 in_bytes = 106024 out_pkts = 1009 out_bytes = 55689 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 860 in_bytes = 49431 out_pkts = 963 out_bytes = 106150 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:07:01:839.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:07:01:839.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:01:839.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692421839306 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:07:01:839.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:07:02:019.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:019.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692422019031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:02:019.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:02:019.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:019.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692422019226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44891 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:02:019.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:02:019.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:02:019.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:02:019.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:019.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692422019818 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:02:019.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:02:613.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:613.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692422613164 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:02:613.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:02:613.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:613.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692422613696 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34169 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:02:613.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:02:614.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:02:614.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:02:614.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:614.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692422614301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:02:614.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:02:614.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:614.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692422614311 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:02:614.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:02:614.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:614.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692422614383 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:02:614.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:02:614.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:614.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692422614461 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:02:614.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:02:615.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:615.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692422615266 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:02:615.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:02:615.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:615.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:02:615.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692422615321 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:02:615.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:02:665.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:665.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692422665502 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:02:665.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:02:666.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:666.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692422666083 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41421 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:02:666.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:02:666.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:02:666.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:02:666.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:666.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692422666512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:02:666.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:02:851.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:07:02:852.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:07:02:852.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:07:02:852.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692422851701 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:02:852.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:02:852.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:07:02:852.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:07:02:852.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:07:02:852.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692422852295 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 635 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 720 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 172 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2439 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 697 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1592, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3182, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:02:852.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:02:852.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:07:02:852.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:07:02:852.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:02:852.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Idle 2025-09-01 Mon 02:07:02:852.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:852.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692422852646 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 636 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 720 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 172 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2439 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 697 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1592, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3184, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:02:852.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:02:853.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:853.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692422852672 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:02:853.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:02:853.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:07:02:853.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:02:853.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692422852760 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 636 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 721 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 172 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2439 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 697 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1593, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3184, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:02:853.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:03:593.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:03:593.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692423593749 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:03:593.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:03:594.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:03:594.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692423594246 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43925 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:03:594.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:03:594.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:03:594.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:03:594.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:03:594.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692423594755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:03:594.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:03:910.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:03:910.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692423909965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:03:910.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:03:910.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:03:910.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692423910446 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36801 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:03:910.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:03:910.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:03:911.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:03:911.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:03:911.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692423911111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:03:911.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:04:127.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:04:127.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692424126973 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:04:127.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:04:127.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:04:127.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692424127508 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36645 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:04:127.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:04:127.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:04:127.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:04:127.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:04:127.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692424127874 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:04:128.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:04:128.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:04:128.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692424127882 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:04:128.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:04:128.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:04:128.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692424127951 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:04:128.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:04:128.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:04:128.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692424128008 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:04:128.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:04:128.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:04:128.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692424128531 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:04:128.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:04:128.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:04:128.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692424128566 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:04:128.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:04:128.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:04:174.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:04:174.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692424174719 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:04:174.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:04:175.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:04:175.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692424175087 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37921 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:04:175.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:04:175.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:04:175.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:04:175.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:04:175.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692424175735 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:04:175.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:129.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:129.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692425129888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:130.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:130.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:130.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692425130385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33937 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:130.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:130.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:05:130.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:05:130.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:130.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692425130877 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:130.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:629.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:629.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692425628949 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:629.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:629.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:629.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692425629156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38273 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:629.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:629.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:05:629.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:05:629.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:629.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692425629542 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:629.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:747.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:747.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692425746964 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:747.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:747.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:747.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692425747163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37797 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:747.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:747.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:05:747.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:05:747.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:747.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692425747529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:747.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:747.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:747.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692425747534 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:747.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:747.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:747.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692425747617 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:05:747.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:05:747.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:747.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692425747723 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:05:747.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:05:747.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:747.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692425747747 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:05:747.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:05:748.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:748.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692425748308 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:748.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:748.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:748.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692425748349 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:05:748.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:05:748.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:05:794.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:794.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692425794511 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:794.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:795.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:795.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692425794992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39521 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:795.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:05:795.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:05:795.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:05:795.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:05:795.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692425795390 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:05:795.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:06:695.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:06:695.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692426695564 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:06:695.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:06:696.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:06:696.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692426696027 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38163 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:06:696.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:06:696.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:06:696.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:06:696.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:06:696.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692426696562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:06:696.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:07:111.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:111.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692427111033 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:07:111.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:07:111.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:111.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692427111225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44495 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:07:111.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:07:111.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:07:111.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:07:111.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:111.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692427111678 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:07:111.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:07:511.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:511.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692427511286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:07:511.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:07:511.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:511.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692427511485 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42807 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:07:511.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:07:512.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:07:512.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:07:512.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:512.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692427512161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:07:512.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:07:512.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:512.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692427512175 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:07:512.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:07:512.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:512.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692427512244 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:07:512.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:07:512.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:512.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692427512505 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:07:512.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:07:512.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:512.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692427512529 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:07:512.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:07:513.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:513.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692427513046 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:07:513.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:07:513.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:07:513.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:513.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692427513086 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:07:513.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:07:562.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:562.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692427562239 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:07:562.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:07:562.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:562.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692427562582 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42757 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:07:562.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:07:562.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:07:562.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:07:562.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:07:563.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692427562989 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:07:563.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:08:517.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:08:517.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692428517143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:08:517.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:08:517.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:08:517.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692428517766 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35719 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:08:517.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:08:518.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:08:518.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:08:518.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:08:518.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692428518121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:08:518.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:08:550.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:08:550.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692428550650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:08:550.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:08:550.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:08:550.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692428550818 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40815 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:08:550.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:08:551.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:08:551.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:08:551.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:08:551.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692428551371 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:08:551.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:09:445.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:09:445.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692429445653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:09:445.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:09:446.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:09:446.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692429446085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33109 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:09:446.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:09:446.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:09:446.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:09:446.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:09:447.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692429446488 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:09:447.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:09:447.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:09:447.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692429446495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:09:447.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:09:447.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:09:447.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692429446563 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:09:447.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:09:447.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:09:447.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:09:447.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692429446624 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:09:447.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:09:447.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:09:447.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692429447154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:09:447.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:09:447.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:09:447.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692429447190 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:09:447.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:09:497.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:09:497.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692429497336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:09:497.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:09:497.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:09:497.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692429497815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40877 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:09:497.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:09:498.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:09:498.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:09:498.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:09:498.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692429498176 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:09:498.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:425.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:425.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692430425331 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:425.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:425.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:425.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692430425815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34063 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:425.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:426.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:10:426.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:10:426.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:426.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692430426238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:426.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:444.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:444.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692430444254 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:444.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:444.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:444.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692430444739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46009 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:444.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:445.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:10:445.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:10:445.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:445.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692430445168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:445.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:855.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:855.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692430855809 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:855.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:856.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:856.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692430856277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45089 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:856.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:856.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:10:856.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:10:856.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:856.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692430856852 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:856.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:856.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:856.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692430856863 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:856.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:856.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:856.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692430856929 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:10:856.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:10:856.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:857.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692430856980 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:10:857.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:10:857.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:857.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692430857582 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:857.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:857.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:857.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:10:857.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692430857620 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:10:857.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:10:902.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:902.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692430902768 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:902.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:903.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:903.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692430903353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38625 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:903.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:10:903.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:10:903.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:10:903.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:10:903.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692430903779 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:10:903.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:11:903.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:11:903.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692431902977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:11:903.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:11:903.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:11:903.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692431903575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46269 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:11:903.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:11:903.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:11:903.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:11:904.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:11:904.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692431904031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:11:904.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:12:151.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:151.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692432151457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:12:151.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:12:151.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:151.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692432151911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35619 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:12:151.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:12:152.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:12:152.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:12:152.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:152.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692432152300 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:12:152.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:12:152.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:152.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692432152306 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:12:152.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:12:152.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:152.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692432152376 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:12:152.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:12:152.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:152.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692432152827 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:12:152.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:12:152.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:152.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692432152853 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:12:152.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:12:153.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:153.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692432153440 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:12:153.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:12:153.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:153.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692432153468 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:12:153.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:12:153.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:12:198.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:198.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692432198653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:12:198.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:12:199.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:199.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692432199220 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44033 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:12:199.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:12:199.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:12:199.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:12:199.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:199.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692432199689 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:12:199.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:12:316.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:316.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692432316428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:12:316.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:12:317.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:317.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692432317027 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40011 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:12:317.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:12:317.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:12:317.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:12:317.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:12:317.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692432317567 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:12:317.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:13:234.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:13:234.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692433234833 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:13:234.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:13:235.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:13:235.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692433235442 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45057 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:13:235.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:13:235.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:13:236.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:13:236.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:13:236.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692433236310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:13:236.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:13:938.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:13:938.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692433938217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:13:938.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:13:938.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:13:938.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692433938742 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44417 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:13:938.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:13:939.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:13:939.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:13:939.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:13:939.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692433939105 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:13:939.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:14:441.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:441.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692434441332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:441.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:14:441.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:441.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692434441809 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37653 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:441.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:14:442.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:14:442.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:14:442.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:442.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692434442340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:442.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:14:442.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:442.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692434442346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:442.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:14:442.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:442.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692434442438 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:14:442.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:14:442.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:442.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692434442566 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:14:442.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:14:442.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:442.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692434442587 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:14:442.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:14:443.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:14:443.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:443.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692434443231 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:443.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:14:443.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:443.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692434443296 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:14:443.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:14:495.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:495.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692434495459 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:495.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:14:495.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:495.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692434495823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33487 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:495.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:14:496.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:14:496.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:14:496.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:496.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692434496154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:496.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:15:144.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:15:144.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692435144411 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:15:144.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:15:144.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:15:144.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692435144586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35311 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:15:144.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:15:144.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:15:144.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:15:145.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:15:145.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692435145004 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:15:145.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:15:504.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:15:504.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692435504330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:15:504.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:15:504.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:15:504.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692435504530 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36609 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:15:504.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:15:504.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:15:504.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:15:504.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:15:504.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692435504915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:15:504.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:16:313.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:16:313.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692436313742 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:16:313.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:16:314.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:16:314.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692436314293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40201 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:16:314.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:16:314.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:16:314.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:16:314.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:16:314.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692436314745 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:16:314.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:16:314.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:16:314.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692436314752 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:16:314.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:16:314.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:16:314.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692436314813 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:16:314.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:16:314.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:16:314.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692436314860 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:16:314.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:16:315.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:16:315.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692436315526 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:16:315.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:16:315.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:16:315.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692436315584 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:16:315.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:16:315.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:16:365.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:16:365.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692436365869 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:16:365.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:16:366.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:16:366.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692436366337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46505 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:16:366.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:16:366.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:16:366.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:16:366.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:16:366.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692436366963 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:16:367.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:16:767.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:38074 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:16:777.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:38076 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:17:348.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:17:348.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692437348085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:17:348.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:17:348.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:17:348.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692437348680 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40929 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:17:348.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:17:349.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:17:349.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:17:349.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:17:349.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692437349140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:17:349.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:17:413.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:17:413.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692437413059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:17:413.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:17:413.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:17:413.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692437413566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36139 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:17:413.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:17:413.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:17:413.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:17:414.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:17:414.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692437413992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:17:414.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:18:043.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:043.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692438043329 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:043.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:18:043.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:043.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692438043525 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44137 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:043.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:18:043.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:18:043.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:18:044.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:044.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692438043998 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:044.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:18:044.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:044.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692438044014 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:044.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:18:044.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:044.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692438044165 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:18:044.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:18:044.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:044.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692438044217 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:18:044.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:18:044.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:044.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692438044769 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:044.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:18:044.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:044.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692438044812 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:18:044.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:18:044.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:18:094.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:094.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692438093973 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:094.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:18:094.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:094.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692438094548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46111 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:094.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:18:094.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:18:094.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:18:094.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:094.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692438094912 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:094.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:033.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:033.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692439033226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:033.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:033.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:033.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692439033413 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43751 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:033.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:033.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:19:033.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:19:033.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:033.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692439033890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:033.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:033.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:033.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692439033899 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:034.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:034.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:034.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692439033977 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:19:034.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:19:034.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:034.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692439034085 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:19:034.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:19:034.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:034.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692439034101 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:19:034.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:19:034.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:034.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:19:034.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692439034738 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:034.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:034.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:035.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692439034786 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:19:035.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:19:084.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:084.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692439083967 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:084.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:084.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:084.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692439084642 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45677 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:084.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:084.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:19:084.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:19:085.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:085.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692439085021 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:085.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:112.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:112.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692439112076 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:112.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:112.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:112.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692439112282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46381 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:112.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:112.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:19:112.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:19:112.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:112.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692439112631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:112.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:455.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:455.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692439455275 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:455.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:455.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:455.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692439455490 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42715 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:455.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:455.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:19:455.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:19:455.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:456.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692439455947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:456.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:20:084.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:20:084.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692440084240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:20:084.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:20:084.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:20:084.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692440084768 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36177 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:20:084.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:20:085.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:20:085.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:20:085.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:20:085.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692440085303 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:20:085.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:164.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:164.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692441164773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:164.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:165.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:165.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692441165217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42481 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:165.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:165.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:21:165.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:21:165.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:165.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692441165590 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:165.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:410.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:410.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692441410261 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:410.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:410.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:410.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692441410923 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39911 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:411.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:411.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:21:411.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:21:411.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:411.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692441411308 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:411.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:411.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:411.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692441411338 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:411.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:411.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:411.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692441411429 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:21:411.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:21:411.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:412.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692441411975 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:21:412.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:21:412.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:412.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692441412022 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:21:412.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:21:412.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:412.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692441412623 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:412.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:412.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:412.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:21:412.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692441412655 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:21:412.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:21:464.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:464.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692441464814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:464.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:465.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:465.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692441465358 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46331 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:465.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:465.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:21:465.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:21:465.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:465.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692441465788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:465.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:903.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:903.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692441903513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:903.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:904.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:904.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692441904055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43995 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:904.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:21:904.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:21:904.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:21:904.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:21:904.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692441904536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:21:904.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:22:429.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:22:429.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692442428971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:22:429.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:22:429.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:22:429.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692442429168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40957 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:22:429.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:22:429.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:22:429.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:22:429.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:22:429.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692442429610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:22:429.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:23:236.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:23:236.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692443236274 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:23:236.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:23:236.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:23:236.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692443236470 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37001 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:23:236.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:23:236.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:23:236.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:23:237.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:23:237.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692443236972 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:23:237.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:23:237.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:23:237.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692443236986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:23:237.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:23:237.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:23:237.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692443237044 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:23:237.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:23:237.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:23:237.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692443237095 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:23:237.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:23:237.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:23:237.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692443237680 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:23:237.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:23:237.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:23:237.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692443237718 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:23:237.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:23:237.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:23:284.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:23:284.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692443284815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:23:284.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:23:285.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:23:285.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692443285322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33593 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:23:285.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:23:285.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:23:286.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:23:286.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:23:286.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692443286035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:23:286.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:24:258.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:24:258.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692444258128 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:24:258.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:24:258.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:24:258.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692444258592 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43065 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:24:258.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:24:258.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:24:259.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:24:259.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:24:259.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692444259100 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:24:259.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:24:373.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:24:373.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692444373763 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:24:373.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:24:374.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:24:374.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692444374273 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42097 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:24:374.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:24:374.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:24:374.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:24:374.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:24:374.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692444374787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:24:374.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:25:125.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:125.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692445125768 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:25:125.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:25:125.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:125.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692445125947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 32893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:25:125.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:25:126.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:25:126.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:25:126.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:127.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692445126271 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:25:127.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:25:127.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:127.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692445126276 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:25:127.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:25:127.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:127.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692445126332 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:25:127.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:25:127.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:127.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692445126382 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:25:127.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:25:127.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:127.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692445126939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:25:127.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:25:127.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:127.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692445126979 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:25:127.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:25:127.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:25:178.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:178.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692445178142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:25:178.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:25:178.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:178.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692445178723 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44413 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:25:178.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:25:179.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:25:179.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:25:179.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:179.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692445179168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:25:179.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:044.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:044.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692446044695 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:044.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:045.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:045.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692446045392 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42801 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:045.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:045.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:26:045.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:26:045.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:045.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692446045805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:045.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:045.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:045.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692446045814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:045.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:045.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:045.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692446045910 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:26:045.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:26:046.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:046.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692446046006 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:26:046.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:26:046.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:046.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692446046023 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:26:046.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:26:046.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:046.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692446046566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:046.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:046.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:046.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692446046621 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:26:046.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:26:046.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:26:077.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:077.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692446077262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:077.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:077.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:077.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692446077773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42017 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:077.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:078.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:26:078.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:26:078.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:078.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692446078302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:078.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:091.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:091.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692446091741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:091.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:092.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:092.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692446092279 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35433 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:092.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:092.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:26:092.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:26:092.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:092.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692446092582 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:092.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:115.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:115.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692446115284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:115.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:115.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:115.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692446115473 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44733 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:115.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:26:115.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:26:115.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:26:115.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:115.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692446115886 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:26:115.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:27:010.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:27:010.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692447010803 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:27:010.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:27:010.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:27:011.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692447010991 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38137 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:27:011.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:27:011.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:27:011.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:27:011.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:27:011.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692447011382 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:27:011.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:118.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:119.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692448118935 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:119.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:119.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:119.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692448119150 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40327 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:119.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:119.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:28:119.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:28:119.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:119.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692448119584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:119.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:119.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:119.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692448119591 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:119.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:119.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:119.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692448119687 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:28:119.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:28:119.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:119.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692448119785 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ]