2025-09-01 Mon 02:15:07:667.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:07:672.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:672.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692907672645 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:15:07:672.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:15:07:672.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:672.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692907672690 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 2230346464 used = 32524 free = 0 buffers = 0 cached = 2303930040 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.315 five_min_avg = 0.14 fifteen_min_avg = 0.125 ] cpu_share = 1.05 ] ] 2025-09-01 Mon 02:15:07:672.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:07:672.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:672.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692907672696 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.05 used_sys_mem = 1991069780 one_min_cpuload = 0.315 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:15:07:672.928 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:15:07:672.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:673.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692907672723 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 2447 in_bytes = 304229 out_tpkts = 2552 out_bytes = 143130 exception_packets = 78 exception_packets_allowed = 65 raw_vhost_stats= [ name = vhost0 in_pkts = 2447 in_bytes = 304229 out_pkts = 2552 out_bytes = 143130 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 2351 in_bytes = 134688 out_pkts = 2449 out_bytes = 304281 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:15:07:673.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:07:673.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:673.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692907672744 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:15:07:673.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:15:07:745.011 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:745.069 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692907744974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:745.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:745.373 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:745.423 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692907745340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40121 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:745.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:745.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:07:745.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:07:746.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:746.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692907745799 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:746.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:746.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:746.746 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692907745806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:746.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:746.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:746.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692907745921 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:07:746.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:07:746.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:746.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692907745999 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:07:746.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:07:746.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:746.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692907746621 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:746.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:746.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:746.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692907746660 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:07:746.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:07:746.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:07:799.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:799.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692907799803 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:799.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:800.385 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:800.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692907800368 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35751 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:800.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:800.701 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:07:800.788 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:07:800.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:800.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692907800816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:800.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:002.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:002.833 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692908002674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:002.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:003.286 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:003.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692908003266 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36715 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:003.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:003.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:08:003.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:08:003.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:003.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692908003653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:003.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:427.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:427.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692908427263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:427.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:427.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:427.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692908427848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39935 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:427.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:428.352 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:08:428.519 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:08:429.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:429.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692908428552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:429.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:429.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:429.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692908428557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:429.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:429.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:429.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692908428637 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:08:429.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:08:429.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:429.768 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692908428728 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:08:429.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:08:429.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:429.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692908428741 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:08:429.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:08:429.819 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:429.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692908429590 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:429.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:429.838 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:429.851 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692908429620 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:08:429.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:08:429.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:08:479.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:479.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692908479835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:479.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:480.029 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:480.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692908480021 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45075 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:480.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:480.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:08:480.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:08:480.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:480.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692908480577 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:480.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:717.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:15:08:717.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:15:08:717.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:08:717.800 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692908717475 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:08:717.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:08:717.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:08:718.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692908717742 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1759 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 154 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1997 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 483 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6787 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1938 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4421, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 8816, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:08:718.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:08:718.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:15:08:718.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:15:08:718.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:15:08:718.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:15:08:718.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:08:718.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Idle 2025-09-01 Mon 02:15:08:718.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:718.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692908718258 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1760 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 155 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1997 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 483 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6787 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1938 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4421, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 8818, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:08:718.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:08:718.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:718.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692908718288 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:08:718.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:08:718.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:15:08:718.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:719.025 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692908718646 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1760 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 155 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1998 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 483 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6787 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1938 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4422, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 8818, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:08:719.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:08:854.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:854.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692908853996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:854.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:854.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:854.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692908854496 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32815 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:854.542 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:854.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:08:855.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:08:855.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:855.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692908855121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:855.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:542.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:542.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692909542706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:542.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:543.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:543.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692909543264 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45653 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:543.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:543.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:09:543.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:09:543.686 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:543.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692909543674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:543.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:127.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:127.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692910127857 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:127.953 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:128.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:128.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692910128451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44709 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:128.519 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:128.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:10:128.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:10:128.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:128.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692910128921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:128.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:997.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:997.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692910997298 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:997.542 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:997.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:997.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692910997452 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39971 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:997.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:997.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:10:997.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:10:997.941 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:997.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692910997922 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:998.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:11:466.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:466.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692911466233 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:11:466.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:11:467.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:467.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692911467130 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46329 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:11:467.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:11:467.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:11:467.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:11:467.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:467.769 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692911467736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:11:467.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:11:467.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:467.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692911467741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:11:467.800 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:11:467.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:467.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692911467790 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:11:467.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:11:467.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:467.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692911467863 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:11:467.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:11:467.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:467.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692911467879 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:11:467.925 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:11:468.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:468.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692911468392 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:11:468.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:11:468.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:468.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692911468432 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:11:468.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:11:468.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:11:487.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:487.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692911487743 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:11:487.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:11:488.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:488.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692911488011 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41229 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:11:488.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:11:488.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:11:488.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:11:488.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:488.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692911488394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:11:488.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:11:521.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:521.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692911521776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:11:521.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:11:522.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:522.329 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692911522305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:11:522.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:11:522.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:11:522.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:11:522.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:11:522.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692911522795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:11:522.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:548.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:549.065 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692912548967 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:550.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:12:550.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:550.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:550.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692912549449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40285 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:550.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:550.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:12:550.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:550.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692912550840 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:550.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:13:981.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:13:981.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692913981010 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:13:981.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:13:981.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:13:981.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692913981839 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41335 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:13:981.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:13:982.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:13:982.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:13:982.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:13:982.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692913982266 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:13:982.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:13:982.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:13:982.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692913982270 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:13:982.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:13:982.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:13:982.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:13:982.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692913982377 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:13:983.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:13:983.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:13:983.037 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692913982417 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:13:983.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:13:983.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:13:983.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692913982859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:13:983.065 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:13:983.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:13:983.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692913982886 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:13:983.092 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:14:031.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:031.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692914031027 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:031.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:032.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:032.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692914032090 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34087 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:032.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:032.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:14:032.549 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:14:032.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:032.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692914032578 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:032.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:369.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:369.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692914369074 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:369.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:369.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:369.695 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692914369619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37257 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:369.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:369.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:14:370.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:14:370.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:370.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692914370061 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:370.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:706.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:706.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692914706064 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:706.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:706.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:706.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692914706539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34329 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:706.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:706.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:14:706.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:14:706.949 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:707.010 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692914706921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:707.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:707.029 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:707.051 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692914706926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:707.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:707.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:707.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692914707133 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:14:707.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:14:707.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:707.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692914707181 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:14:707.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:14:707.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:707.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692914707608 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:707.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:707.694 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:707.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692914707641 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:14:707.721 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:14:707.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:14:755.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:755.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692914755212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:755.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:755.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:755.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692914755366 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39557 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:755.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:755.675 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:14:755.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:14:755.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:755.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692914755805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:755.856 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:022.799 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:022.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692915022769 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:022.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:023.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:023.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692915023343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46201 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:023.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:023.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:15:023.726 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:15:023.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:023.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692915023761 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:023.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:268.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:268.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692915268551 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:268.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:268.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:268.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692915268703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42571 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:268.746 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:268.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:15:269.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:15:269.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:269.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692915269113 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:269.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:269.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:269.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692915269119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:269.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:269.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:269.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692915269174 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:15:269.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:15:269.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:269.501 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692915269459 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:15:269.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:15:269.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:269.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692915269478 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:15:269.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:15:269.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:269.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692915269950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:270.007 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:270.017 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:270.037 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:15:270.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692915269992 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:15:270.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:15:320.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:320.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692915320204 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:320.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:320.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:320.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692915320406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43009 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:320.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:320.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:15:320.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:15:320.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:320.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692915320843 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:320.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:674.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:674.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692915674004 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:674.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:674.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:674.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692915674460 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39331 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:674.504 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:674.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:15:674.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:15:675.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:675.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692915675009 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:675.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:320.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:320.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692916319989 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:320.190 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:320.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:320.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692916320553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42193 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:320.623 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:320.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:16:320.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:16:321.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:321.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692916321020 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:321.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:057.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:057.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692917057886 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:058.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:058.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:058.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692917058107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43039 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:058.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:058.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:17:058.542 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:17:058.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:058.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692917058588 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:058.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:493.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:493.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692917493094 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:493.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:493.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:493.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692917493526 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34437 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:493.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:493.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:17:494.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:17:494.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:494.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692917494053 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:494.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:229.190 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:229.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692918229154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:229.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:229.701 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:229.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692918229693 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45619 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:229.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:230.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:18:230.179 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:18:230.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:230.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692918230206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:230.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:618.250 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:618.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692918618221 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:618.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:618.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:618.416 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692918618393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37161 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:618.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:618.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:18:618.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:18:618.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:619.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692918618983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:619.027 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:619.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:619.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692918618990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:619.051 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:619.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:619.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692918619063 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:18:619.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:18:619.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:619.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692918619156 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:18:619.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:18:619.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:619.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692918619173 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:18:619.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:18:619.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:619.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692918619665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:619.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:619.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:619.769 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692918619699 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:18:619.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:18:619.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:18:665.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:665.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692918664941 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:665.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:665.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:665.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692918665448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34307 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:665.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:665.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:18:665.799 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:18:665.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:665.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692918665831 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:665.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:19:593.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:19:593.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692919592976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:19:593.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:19:593.228 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:19:593.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692919593143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45015 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:19:593.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:19:593.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:19:593.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:19:593.646 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:19:593.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692919593629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:19:593.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:20:803.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:803.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692920802880 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:20:803.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:20:804.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:804.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692920804141 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43415 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:20:804.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:20:804.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:20:804.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:20:805.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:805.622 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692920804665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:20:805.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:20:805.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:805.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692920804670 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:20:805.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:20:805.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:805.699 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692920804782 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:20:805.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:20:805.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:805.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692920804845 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:20:805.761 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:20:805.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:805.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692920805540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:20:805.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:20:805.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:805.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692920805574 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:20:805.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:20:805.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:20:852.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:852.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692920851971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:20:852.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:20:852.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:852.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692920852872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39403 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:20:853.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:20:853.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:20:853.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:20:853.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:853.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692920853286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:20:853.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:454.228 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:454.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692921454188 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:454.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:454.768 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:454.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692921454760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38605 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:454.796 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:455.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:21:455.194 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:21:455.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:455.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692921455228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:455.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:455.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:455.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692921455235 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:455.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:455.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:455.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692921455337 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:21:455.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:21:455.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:455.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692921455395 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:21:455.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:21:456.016 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:456.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692921455981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:456.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:456.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:456.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692921456022 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:21:456.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:21:456.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:21:465.742 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:465.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692921465717 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:465.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:466.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:466.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692921466149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36677 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:466.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:466.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:21:466.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:21:466.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:466.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692921466547 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:466.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:501.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:501.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692921501140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:501.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:502.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:502.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692921502037 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40961 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:502.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:502.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:21:502.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:21:502.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:502.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692921502339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:502.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:915.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:915.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692921915439 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:915.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:915.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:915.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692921915604 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33499 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:915.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:915.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:21:915.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:21:916.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:916.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692921916031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:916.081 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:082.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:082.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692922082378 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:082.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:082.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:082.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:22:082.984 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692922082543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44333 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:083.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:083.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:22:083.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:083.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692922083135 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:083.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:083.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:083.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692922083151 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:083.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:083.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:083.280 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692922083251 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:22:083.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:22:083.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:083.351 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692922083325 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:22:083.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:22:083.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:083.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692922083340 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:22:083.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:22:083.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:083.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692922083817 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:083.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:083.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:083.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692922083845 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:22:083.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:22:083.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:22:131.065 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:131.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692922131032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:131.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:131.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:131.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692922131598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39607 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:131.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:131.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:22:131.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:22:132.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:132.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692922132015 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:132.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:429.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:429.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692922429467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:429.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:429.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:430.019 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692922429952 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35675 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:430.026 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:430.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:22:430.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:22:430.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:430.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692922430436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:430.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:059.185 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:059.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692923059095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:059.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:059.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:059.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692923059945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41541 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:060.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:060.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:23:060.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:23:060.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:060.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692923060453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:060.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:356.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:49160 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:23:365.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:49162 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:24:040.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:040.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692924040174 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:040.488 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:040.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:040.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692924040396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33087 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:040.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:040.725 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:24:040.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:24:040.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:040.872 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692924040839 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:040.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:500.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:500.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692924500570 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:500.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:501.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:501.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692924501455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42157 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:501.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:501.821 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:24:501.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:24:501.949 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:502.024 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692924501936 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:502.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:932.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:932.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692924932595 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:932.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:933.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:933.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692924933575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44551 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:933.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:933.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:24:934.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:24:934.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:934.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692924934102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:934.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:534.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:534.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692925534679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:534.841 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:535.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:535.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692925535170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40029 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:535.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:535.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:25:535.519 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:25:535.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:535.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692925535572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:535.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:535.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:535.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692925535581 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:535.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:535.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:535.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692925535652 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:25:536.013 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:25:536.025 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:536.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692925535734 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:25:536.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:25:536.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:536.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692925535751 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:25:536.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:25:536.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:536.283 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692925536250 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:536.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:536.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:536.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692925536296 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:25:536.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:25:536.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:25:586.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:586.595 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692925586438 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:586.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:587.113 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:587.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692925587095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34641 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:587.145 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:587.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:25:587.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:25:587.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:587.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692925587562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:587.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:26:514.812 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:26:514.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692926514700 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:26:514.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:26:515.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:26:515.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692926515724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39317 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:26:515.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:26:516.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:26:516.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:26:516.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:26:516.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692926516185 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:26:516.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:27:749.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:749.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692927748987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:27:749.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:27:749.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:749.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692927749921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42493 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:27:749.968 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:27:750.316 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:27:750.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:27:750.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:750.491 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692927750441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:27:750.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:27:750.527 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:750.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692927750448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:27:750.549 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:27:750.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:750.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692927750524 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:27:750.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:27:750.614 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:750.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692927750564 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:27:750.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:27:751.072 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:751.095 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692927751028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:27:751.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:27:751.110 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:751.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692927751060 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:27:751.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:27:751.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:27:796.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:796.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692927796222 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:27:796.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:27:796.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:796.839 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692927796803 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38857 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:27:796.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:27:797.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:27:797.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:27:797.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:797.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692927797286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:27:797.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:478.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:478.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692928478321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:478.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:479.519 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:479.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692928479503 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36069 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:479.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:479.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:28:479.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:28:479.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:480.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692928479951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:480.013 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:678.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:678.286 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692928678094 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:678.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:678.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:678.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692928678732 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:678.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:679.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:28:679.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:28:679.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:679.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:28:679.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692928679165 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:679.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:679.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:679.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692928679174 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:679.988 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:679.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:680.025 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692928679228 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:28:680.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:28:680.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:680.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692928679275 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:28:680.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:28:680.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:680.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692928679749 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:680.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:680.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:680.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692928679787 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:28:680.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:28:730.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:731.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692928730930 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:731.054 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:731.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:731.515 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692928731445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45247 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:731.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:731.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:28:731.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:28:731.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:731.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692928731870 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:731.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:786.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:786.355 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692928786261 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:786.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:786.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:787.025 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692928786919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36115 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:787.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:787.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:28:787.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:28:787.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:787.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692928787330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:787.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:787.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:787.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692928787346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:787.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:787.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:787.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692928787424 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:28:787.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:28:787.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:787.549 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692928787515 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:28:787.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:28:787.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:787.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692928787532 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:28:787.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:28:788.351 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:788.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692928788270 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:788.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:788.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:788.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692928788322 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:28:788.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:28:788.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:28:836.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:836.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692928836653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:836.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:836.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:836.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692928836828 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 32889 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:836.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:837.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:28:837.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:28:837.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:837.352 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692928837302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:837.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:859.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:859.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692928859449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:859.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:859.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:859.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692928859658 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43839 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:859.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:860.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:28:860.111 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:28:860.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:860.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692928860144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:860.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:785.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:785.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692929785008 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:785.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:785.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:785.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692929785614 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36461 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:785.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:785.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:29:786.022 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:29:786.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:786.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692929786074 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:786.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:863.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:863.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692929863446 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:863.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:864.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:864.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692929864065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37361 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:864.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:864.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:29:864.460 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:29:864.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:864.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692929864517 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:864.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:002.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:002.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692931002349 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:002.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:002.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:002.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692931002876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38589 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:002.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:003.291 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:31:003.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:31:003.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:003.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692931003438 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:003.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:712.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:712.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692931712185 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:712.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:713.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:713.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692931713132 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41093 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:713.179 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:713.521 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:31:713.608 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:31:713.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:713.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692931713639 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:713.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:772.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:772.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692931772642 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:772.746 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:773.022 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:773.069 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692931773001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46593 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:773.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:773.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:31:773.416 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:31:773.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:773.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692931773447 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:773.499 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:080.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:080.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692932080111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:080.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:080.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:080.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692932080587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43305 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:080.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:080.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:32:081.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:32:081.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:081.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692932081079 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:081.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:081.128 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:081.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692932081084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:081.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:081.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:081.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692932081132 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:32:081.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:32:081.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:081.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692932081205 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:32:081.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:32:081.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:081.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692932081221 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:32:081.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:32:081.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:081.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692932081687 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:081.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:081.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:081.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692932081723 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:32:081.802 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:32:081.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:32:133.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:133.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692932133009 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:133.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:133.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:133.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692932133805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:133.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:134.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:32:135.056 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:32:135.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:135.145 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692932135084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:135.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:33:080.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:33:080.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692933080206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:33:080.373 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:33:080.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:33:080.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692933080673 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:33:080.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:33:081.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:33:081.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:33:081.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:33:081.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692933081190 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:33:081.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:34:953.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:34:953.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692934953320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:34:953.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:34:954.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:34:954.316 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692934954266 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39745 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:34:954.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:34:954.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:34:954.727 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:34:954.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:34:954.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692934954755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:34:954.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:215.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:215.721 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692935215561 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:215.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:216.460 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:216.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:35:216.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:35:217.019 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692935216448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39781 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:217.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:217.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:217.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692935216890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:217.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:217.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:217.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692935216894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:217.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:217.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:217.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692935216965 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:35:217.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:217.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:217.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692935217043 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:35:217.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:217.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:217.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692935217634 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:217.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:217.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:217.728 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:35:217.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692935217708 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:35:217.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:270.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:270.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692935270837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:270.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:271.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:271.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692935271376 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38283 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:271.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:271.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:35:271.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:35:271.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:271.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692935271935 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:271.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:313.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:313.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692935313772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:313.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:314.431 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:314.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692935314408 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34839 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:314.487 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:314.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:35:314.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:35:314.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:314.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692935314818 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:314.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:315.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:35:315.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:315.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692935314842 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:315.667 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:315.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:315.699 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692935314896 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:35:315.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:315.726 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:315.742 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692935314966 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:35:315.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:315.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:315.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692935315415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:315.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:315.779 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:315.796 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692935315453 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:35:315.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:363.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:363.720 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692935363623 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:363.732 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:363.796 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:363.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692935363788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33307 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:363.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:364.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:35:364.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:35:364.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:364.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692935364218 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:364.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:445.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:445.720 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692935445628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:445.730 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:445.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:445.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692935445796 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42897 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:445.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:446.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:35:446.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:35:446.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:446.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692935446168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:446.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:446.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:446.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692935446175 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:446.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:446.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:446.274 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692935446248 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:35:446.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:446.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:446.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692935446326 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:35:446.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:446.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:446.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692935446341 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:35:446.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:446.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:446.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692935446820 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:446.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:446.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:446.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692935446852 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:35:446.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:446.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:35:493.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:493.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692935493115 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:493.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:493.527 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:493.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692935493519 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41517 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:493.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:493.819 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:35:494.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:35:494.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:494.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692935494104 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:494.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:253.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:253.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692936253082 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:253.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:254.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:254.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692936254035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33209 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:254.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:254.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:36:254.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:36:254.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:254.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692936254511 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:254.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:399.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:399.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692936399345 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:399.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:400.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:400.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692936400222 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43569 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:400.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:400.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:36:400.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:36:400.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:400.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692936400704 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:400.755 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:421.306 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:421.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692936421214 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:421.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:422.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:422.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692936422093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40429 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:422.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:422.447 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:36:423.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:36:423.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:423.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692936423159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:423.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:667.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:667.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692937667255 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 32 dequeues = 32 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 9107 dequeues = 9107 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 33 dequeues = 33 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 33 dequeues = 33 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 9207 dequeues = 9207 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692908718600 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692936423190 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:15:37:667.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:37:667.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:667.623 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692937667347 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1817 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 157 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2066 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 499 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7017 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2004 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4572, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 9107, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:37:667.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:37:672.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:673.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692937672896 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.25, ] ] ] 2025-09-01 Mon 02:15:37:673.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:15:37:673.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:673.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692937672940 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 2541 in_bytes = 321182 out_tpkts = 2650 out_bytes = 148932 exception_packets = 80 exception_packets_allowed = 67 raw_vhost_stats= [ name = vhost0 in_pkts = 2541 in_bytes = 321182 out_pkts = 2650 out_bytes = 148932 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] total_in_bandwidth_utilization = 6.86842e-15 total_out_bandwidth_utilization = 1.60787e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1267, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 2966, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 2446 in_bytes = 140364 out_pkts = 2543 out_bytes = 321234 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:15:37:673.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:37:673.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:673.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692937673194 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:15:37:673.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:15:38:259.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:259.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692938259295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:259.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:259.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:259.911 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692938259836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40351 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:259.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:260.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:38:260.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:38:260.228 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:260.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692938260193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:260.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:342.667 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:342.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692938342631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:342.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:343.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:343.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692938343172 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44887 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:343.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:343.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:38:343.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:38:343.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:343.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692938343560 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:343.620 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:362.866 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:362.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692938362828 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:362.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:363.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:363.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692938363337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46413 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:363.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:363.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:38:363.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:38:363.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:363.728 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692938363682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:363.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:699.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:699.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692938698898 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:699.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:699.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:699.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692938699509 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34967 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:699.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:699.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:38:699.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:38:700.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:700.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692938700059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:700.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:700.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:700.316 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692938700068 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:700.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:700.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:700.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692938700159 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:38:700.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:38:700.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:700.567 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692938700448 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:38:700.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:38:700.589 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:700.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692938700465 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:38:700.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:38:701.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:701.671 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692938701587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:701.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:38:701.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:701.825 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:701.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692938701648 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:38:701.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:38:718.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:15:38:718.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:15:38:719.115 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:38:719.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692938718795 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:38:719.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:38:719.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:38:719.291 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692938719095 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1821 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 159 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2070 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 500 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7031 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2008 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4581, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 9130, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:38:719.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:38:719.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:15:38:719.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:15:38:719.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:15:38:719.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:15:38:720.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:38:720.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Idle 2025-09-01 Mon 02:15:38:720.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:720.277 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692938720001 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1822 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 160 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2070 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 500 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7031 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2008 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4581, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 9132, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:38:720.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:38:720.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:720.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692938720027 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:38:720.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:38:720.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:15:38:720.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:720.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692938720124 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1822 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 160 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2071 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 500 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7031 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2008 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4582, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 9132, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:38:720.515 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:38:749.817 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:749.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692938749771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:749.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:750.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:750.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692938750372 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39293 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:750.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:750.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:38:750.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:38:750.727 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:750.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692938750718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:750.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:39:686.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:39:687.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692939686858 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:39:687.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:39:687.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:39:687.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692939687332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34493 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:39:687.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:39:687.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:39:687.696 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:39:687.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:39:687.799 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692939687720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:39:687.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:41:739.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:41:740.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692941739848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:41:740.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:41:740.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:41:740.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692941740640 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38853 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:41:740.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:41:740.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:41:741.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:41:741.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:41:741.141 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692941741101 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:41:741.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:159.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:159.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692942159669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:159.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:160.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:160.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692942160648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35673 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:160.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:161.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:42:161.110 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:42:161.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:161.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942161133 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:161.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:161.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:161.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942161139 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:161.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:161.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:161.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942161261 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:42:161.824 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:42:161.833 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:161.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942161306 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:42:161.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:42:161.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:161.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942161739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:161.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:161.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:161.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942161761 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:42:161.908 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:42:161.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:42:184.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:184.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692942184321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:184.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:184.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:184.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692942184633 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35023 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:184.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:184.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:42:184.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:42:185.029 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:185.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942185010 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:185.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:185.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:185.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942185023 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:185.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:185.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:185.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942185069 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:42:185.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:42:185.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:185.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942185138 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:42:185.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:42:185.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:185.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942185153 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:42:185.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:42:185.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:185.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942185648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:185.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:185.696 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:185.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942185675 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:42:185.720 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:42:185.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:42:207.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:207.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692942207046 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:207.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:207.504 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:207.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692942207487 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34897 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:207.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:207.802 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:42:207.839 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:42:207.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:207.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692942207873 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:207.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:232.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:232.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692942231855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:232.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:232.067 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:232.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692942231988 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41301 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:232.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:232.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:42:232.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:42:232.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:232.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692942232420 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:232.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:287.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:287.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692942287830 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:288.011 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:288.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:288.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692942288510 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35261 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:288.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:288.772 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:42:288.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:42:288.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:288.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692942288969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:289.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:289.009 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:289.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692942288977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:289.024 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:289.031 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:289.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692942289027 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:42:289.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:42:289.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:289.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692942289069 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:42:289.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:42:289.567 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:289.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692942289535 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:289.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:289.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:289.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692942289579 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:42:289.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:42:289.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:42:338.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:339.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692942338900 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:339.019 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:339.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:339.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692942339327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38667 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:339.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:339.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:42:339.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:42:339.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:42:339.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692942339737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:42:339.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:43:207.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:43:207.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692943207006 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:43:207.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:43:207.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:43:207.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692943207288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46295 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:43:207.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:43:208.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:43:208.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:43:208.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:43:208.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692943208121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:43:208.256 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:43:266.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:43:267.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692943266859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:43:267.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:43:267.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:43:267.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692943267395 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45069 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:43:267.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:43:267.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:43:267.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:43:267.774 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:43:267.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692943267724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:43:267.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:43:276.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:43:276.968 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692943276783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:43:276.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:43:276.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:43:277.021 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692943276901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42291 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:43:277.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:43:277.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:43:277.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:43:277.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:43:277.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692943277463 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:43:277.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:203.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:203.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692945203571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:203.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:203.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:203.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692945203715 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39473 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:203.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:204.019 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:45:204.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:45:204.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:204.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692945204105 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:204.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:211.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:211.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692945211799 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:211.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:212.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:212.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692945212330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35503 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:212.385 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:212.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:45:212.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:45:212.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:212.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692945212703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:212.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:242.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:242.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692945242205 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:242.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:242.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:242.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692945242790 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41541 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:242.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:243.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:45:243.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:45:243.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:243.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692945243192 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:243.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:377.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:377.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692945377194 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:377.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:377.934 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:377.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692945377927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44683 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:377.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:378.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:45:379.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:45:379.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:380.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692945379490 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:380.851 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:380.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:380.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692945379495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:380.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:380.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:380.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:45:380.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692945379570 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:45:381.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:45:381.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:381.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692945379661 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:45:381.047 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:45:381.054 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:381.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692945379714 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:45:381.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:45:381.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:381.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692945380817 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:381.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:381.110 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:381.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692945380857 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:45:381.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:45:426.167 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:426.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692945426049 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:426.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:427.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:427.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692945427157 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41123 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:427.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:45:427.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:45:427.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:45:427.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:45:427.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692945427521 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:45:427.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:46:489.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:46:489.772 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692946489621 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:46:489.782 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:46:490.514 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:46:490.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692946490504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38435 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:46:490.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:46:490.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:46:490.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:46:490.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:46:490.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692946490830 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:46:490.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:47:324.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.57:8085::10.0.0.57:57380 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:48:579.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:48:579.141 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692948578978 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:48:579.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:48:579.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:48:580.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692948579964 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38193 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:48:580.016 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:48:580.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:48:580.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:48:580.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:48:580.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692948580324 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:48:580.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:164.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:164.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692949164235 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:164.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:164.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:164.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692949164915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40383 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:164.965 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:165.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:49:165.256 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:49:165.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:165.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692949165292 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:165.373 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:165.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:165.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692949165297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:165.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:166.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:166.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692949165432 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:49:166.189 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:49:166.198 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:166.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692949165606 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:49:166.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:49:166.234 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:166.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692949165624 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:49:166.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:49:166.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:166.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692949166089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:166.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:166.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:166.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692949166111 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:49:166.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:49:166.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:49:213.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:213.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692949213230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:213.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:214.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:214.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692949214418 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37599 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:214.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:214.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:49:214.819 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:49:214.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:214.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692949214848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:214.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:347.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:347.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692949347297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:347.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:348.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:348.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692949348060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43153 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:348.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:348.385 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:49:348.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:49:348.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:348.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692949348436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:348.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:348.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:348.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692949348440 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:348.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:348.796 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:348.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692949348529 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:49:348.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:49:348.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:348.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692949348577 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:49:348.892 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:49:349.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:349.062 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692949349032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:349.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:349.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:349.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:49:349.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692949349059 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:49:349.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:49:399.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:399.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692949399193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:399.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:399.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:399.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692949399677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45517 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:399.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:399.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:49:399.988 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:49:400.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:400.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692949400010 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:400.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:460.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:460.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692949460800 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:460.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:461.667 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:461.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692949461654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40317 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:461.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:461.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:49:462.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:49:462.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:462.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692949462061 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:462.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:462.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:462.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692949462068 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:462.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:462.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:462.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692949462216 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:49:462.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:49:462.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:462.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692949462270 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:49:462.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:49:462.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:462.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692949462779 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:462.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:462.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:462.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692949462815 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:49:463.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:49:463.026 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:49:516.219 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:516.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692949516125 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:516.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:517.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:517.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692949517062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37813 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:517.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:49:517.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:49:518.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:49:518.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:49:518.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692949518504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:49:518.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog