2026-09-14 Mon 02:36:19:600.617 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:36:19:601.673 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:45724::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:36:19:601.714 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:45724::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:36:19:601.724 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:45724::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:36:19:601.806 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:36:19:601.838 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:36:19:601.872 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit 2026-09-14 Mon 02:36:19:601.891 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 12 2026-09-14 Mon 02:36:19:601.972 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:19:601.981 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:36:19:601.997 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: SANDESH: Send FAILED: 1789353379601864 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:36:19:602.004 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:36:19:602.012 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:19:602.018 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:36:19:602.030 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:19:602.035 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:36:19:602.189 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: SANDESH: Send FAILED: 1789353379601958 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 12 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 71578 calls = 36 average_bytes = 1988 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 308 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 933 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 60 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 40 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1394, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 150, _iter59->first = sent _iter59->second = 51, ] ] ] 2026-09-14 Mon 02:36:19:602.222 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:36:23:386.115 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:36:23:399.278 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:36:23:399.687 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:36:23:399.737 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:36:27:729.810 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:43615 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:36:27:729.908 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:36:27:729.960 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:43615 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:36:27:730.040 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:27:730.043 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:36:27:730.088 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:36:27:730.168 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:36:27:730.297 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:27:730.335 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:36:27:730.367 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 43615 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:36:27:731.783 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:27:731.800 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:36:27:731.816 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:27:731.826 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:36:27:731.836 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:27:731.843 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:36:27:731.869 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:36:27:731.946 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:36:27:731.955 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:36:27:732.029 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:36:28:400.108 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:36:28:413.151 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:36:28:413.479 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:36:28:413.496 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:36:33:413.871 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:36:33:426.688 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:36:33:427.025 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:36:33:427.043 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:36:37:542.617 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:37:542.667 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:36:37:542.743 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: SANDESH: Send FAILED: 1789353397542474 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 209 dequeues = 209 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 259 dequeues = 259 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353379601871 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353387731834 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 6 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 8345 calls = 4 average_bytes = 2086 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:36:37:542.765 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:36:37:542.776 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:37:542.782 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:36:37:542.943 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: SANDESH: Send FAILED: 1789353397542557 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 12 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 79923 calls = 40 average_bytes = 1998 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 311 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 942 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 57 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 45 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1411, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 152, _iter59->first = sent _iter59->second = 57, ] ] ] 2026-09-14 Mon 02:36:37:542.961 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:36:38:427.444 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:36:38:441.089 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:36:38:441.591 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:36:38:441.605 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:36:43:441.955 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:36:43:455.321 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:36:43:455.864 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:36:43:455.886 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:36:48:456.267 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:36:48:469.309 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:36:48:469.658 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:36:48:469.728 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:36:49:602.063 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: 10.0.0.185:8086 EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:36:49:602.190 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:36:49:602.247 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit : EvConnectTimerExpired 2026-09-14 Mon 02:36:49:602.413 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Return to idle with 1000 2026-09-14 Mon 02:36:49:602.424 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Idle 2026-09-14 Mon 02:36:49:602.535 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:36:49:602.560 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: SANDESH: Send FAILED: 1789353409602297 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:36:49:602.581 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:36:49:602.590 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvTcpDeleteSession in state Idle 2026-09-14 Mon 02:36:49:602.635 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:36:49:602.810 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: SANDESH: Send FAILED: 1789353409602518 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 12 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 79923 calls = 40 average_bytes = 1998 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 314 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 951 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 59 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 45 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1423, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 154, _iter59->first = sent _iter59->second = 57, ] ] ] 2026-09-14 Mon 02:36:49:602.832 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:36:50:603.047 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-09-14 Mon 02:36:50:603.450 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Connect : Start Connect timer 10.0.0.185:8086 2026-09-14 Mon 02:36:50:603.580 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:36:50:603.617 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: SANDESH: Send FAILED: 1789353410603196 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:36:50:603.629 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:36:50:603.641 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:36:50:603.830 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: SANDESH: Send FAILED: 1789353410603545 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 12 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 79923 calls = 40 average_bytes = 1998 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 314 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 951 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 45 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1423, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 156, _iter59->first = sent _iter59->second = 57, ] ] ] 2026-09-14 Mon 02:36:50:603.859 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:36:50:605.145 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:49252::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:36:50:605.172 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:49252::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:36:50:605.180 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:49252::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:36:50:605.268 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:36:50:605.317 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:36:50:605.365 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit 2026-09-14 Mon 02:36:50:605.404 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 13 2026-09-14 Mon 02:36:50:605.500 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:50:605.509 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:36:50:605.529 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: SANDESH: Send FAILED: 1789353410605356 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:36:50:605.536 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:36:50:605.544 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:50:605.549 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:36:50:605.562 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:50:605.567 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:36:50:605.711 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: SANDESH: Send FAILED: 1789353410605484 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 13 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 79923 calls = 40 average_bytes = 1998 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 314 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 951 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 62 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 45 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1424, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 158, _iter59->first = sent _iter59->second = 57, ] ] ] 2026-09-14 Mon 02:36:50:605.740 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:36:53:470.091 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:36:53:483.562 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:36:53:483.908 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:36:53:483.923 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:36:58:484.287 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:36:58:497.383 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:36:58:497.792 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:36:58:497.812 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:36:58:791.874 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:46267 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:36:58:791.939 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:36:58:791.988 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:46267 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:36:58:792.102 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:36:58:792.103 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:58:792.169 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:36:58:792.186 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:58:792.233 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:36:58:792.273 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:36:58:792.345 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 46267 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:36:58:793.794 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:58:793.830 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:36:58:793.845 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:58:793.852 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:36:58:793.875 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:36:58:793.937 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:36:58:793.943 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:36:58:794.022 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:36:58:794.033 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:36:58:794.078 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:37:03:498.230 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:03:516.474 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:03:519.085 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:03:519.107 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:37:07:542.986 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:07:543.074 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:37:07:543.191 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353427542828 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 223 dequeues = 223 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 277 dequeues = 277 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353410605365 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353418794020 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 6 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 8349 calls = 4 average_bytes = 2087 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:37:07:543.246 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:37:07:543.259 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:07:543.266 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:37:07:543.406 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353427542914 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 13 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 88272 calls = 44 average_bytes = 2006 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 317 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 960 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 63 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 50 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1441, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 160, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-09-14 Mon 02:37:07:543.423 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:37:08:519.625 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:08:533.816 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:08:534.310 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:08:534.363 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:37:13:534.750 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:13:548.680 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:13:549.038 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:13:549.057 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:37:18:549.347 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:18:563.503 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:18:563.901 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:18:563.952 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:37:20:605.567 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: 10.0.0.185:8086 EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:37:20:605.667 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:37:20:605.703 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: ClientInit : EvConnectTimerExpired 2026-09-14 Mon 02:37:20:605.844 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Return to idle with 1000 2026-09-14 Mon 02:37:20:605.855 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Idle 2026-09-14 Mon 02:37:20:605.958 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:37:20:605.997 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: SANDESH: Send FAILED: 1789353440605745 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:37:20:606.005 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:37:20:606.014 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Processing scm::EvTcpDeleteSession in state Idle 2026-09-14 Mon 02:37:20:606.023 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:37:20:606.192 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: SANDESH: Send FAILED: 1789353440605941 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 13 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 88272 calls = 44 average_bytes = 2006 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 320 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 969 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 63 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 50 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1453, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 162, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-09-14 Mon 02:37:20:606.230 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:37:21:607.256 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-09-14 Mon 02:37:21:609.733 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:33644::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:37:21:609.759 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:33644::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:37:21:609.770 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:33644::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:37:21:609.806 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Connect : Start Connect timer 10.0.0.185:8086 2026-09-14 Mon 02:37:21:609.921 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:37:21:609.945 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: SANDESH: Send FAILED: 1789353441607716 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:37:21:609.953 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:37:21:609.965 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:37:21:609.974 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:37:21:610.010 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: ClientInit 2026-09-14 Mon 02:37:21:610.036 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 14 2026-09-14 Mon 02:37:21:610.105 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:21:610.113 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:37:21:610.281 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: SANDESH: Send FAILED: 1789353441609889 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 13 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 88272 calls = 44 average_bytes = 2006 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 320 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 969 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 64 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 50 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1454, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 164, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-09-14 Mon 02:37:21:610.315 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:37:21:610.324 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:21:610.330 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:37:21:610.344 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: SANDESH: Send FAILED: 1789353441610003 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:37:21:610.350 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:37:21:610.357 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:21:610.362 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:37:21:610.372 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:21:610.378 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:37:21:610.502 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: SANDESH: Send FAILED: 1789353441610087 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 14 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 88272 calls = 44 average_bytes = 2006 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 320 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 969 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 64 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 50 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1454, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 165, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-09-14 Mon 02:37:21:610.518 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:37:23:564.342 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:23:577.742 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:23:578.137 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:23:578.154 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:37:26:070.016 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:43199 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:37:26:070.094 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:37:26:070.152 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:43199 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:37:26:070.261 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:37:26:070.265 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:26:070.346 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:37:26:070.388 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:37:26:070.457 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:26:070.469 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:37:26:070.492 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 43199 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:37:26:071.836 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:26:071.849 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:37:26:071.870 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:37:26:071.928 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:26:071.937 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:37:26:071.945 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:26:071.951 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:37:26:072.040 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:37:26:072.055 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:37:26:072.125 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:37:28:578.491 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:28:592.509 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:28:592.873 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:28:592.918 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:37:33:593.299 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:33:606.336 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:33:606.703 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:33:606.717 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:37:37:543.303 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:37:543.359 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:37:37:543.440 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: SANDESH: Send FAILED: 1789353457543136 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 237 dequeues = 237 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 295 dequeues = 295 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353441610009 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353446071944 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 6 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 8350 calls = 5 average_bytes = 1670 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:37:37:543.465 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:37:37:543.475 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:37:543.482 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:37:37:543.619 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: SANDESH: Send FAILED: 1789353457543235 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 14 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 96622 calls = 49 average_bytes = 1971 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 323 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 978 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 14 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 67 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 55 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1471, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 168, _iter59->first = sent _iter59->second = 69, ] ] ] 2026-09-14 Mon 02:37:37:543.637 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:37:38:607.081 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:38:620.131 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:38:620.495 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:38:620.564 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:37:43:621.025 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:43:634.798 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:43:635.290 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:43:635.310 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:37:48:635.682 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:48:651.440 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:48:651.855 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:48:651.874 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:37:51:610.167 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552799446592, Pid 64205]: 10.0.0.185:8086 EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:37:51:610.281 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:37:51:610.333 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: ClientInit : EvConnectTimerExpired 2026-09-14 Mon 02:37:51:610.487 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Return to idle with 1000 2026-09-14 Mon 02:37:51:610.515 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Idle 2026-09-14 Mon 02:37:51:610.633 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:37:51:610.661 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: SANDESH: Send FAILED: 1789353471610379 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:37:51:610.670 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:37:51:610.679 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Processing scm::EvTcpDeleteSession in state Idle 2026-09-14 Mon 02:37:51:610.721 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:37:51:610.907 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: SANDESH: Send FAILED: 1789353471610612 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 14 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 96622 calls = 49 average_bytes = 1971 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 326 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 987 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 14 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 69 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 55 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1483, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 170, _iter59->first = sent _iter59->second = 69, ] ] ] 2026-09-14 Mon 02:37:51:610.932 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:37:52:610.649 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-09-14 Mon 02:37:52:610.958 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Connect : Start Connect timer 10.0.0.185:8086 2026-09-14 Mon 02:37:52:611.072 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:37:52:611.099 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: SANDESH: Send FAILED: 1789353472610746 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:37:52:611.108 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:37:52:611.116 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:37:52:611.296 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: SANDESH: Send FAILED: 1789353472611056 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 14 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 96622 calls = 49 average_bytes = 1971 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 326 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 987 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 14 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 55 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1483, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 172, _iter59->first = sent _iter59->second = 69, ] ] ] 2026-09-14 Mon 02:37:52:611.327 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:37:52:612.728 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:47180::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:37:52:612.773 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:47180::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:37:52:612.785 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:47180::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:37:52:612.850 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:37:52:612.870 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:37:52:612.897 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit 2026-09-14 Mon 02:37:52:612.918 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 15 2026-09-14 Mon 02:37:52:612.986 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:52:612.995 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:37:52:613.010 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: SANDESH: Send FAILED: 1789353472612891 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:37:52:613.017 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:37:52:613.025 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:52:613.031 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:37:52:613.050 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:52:613.062 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:37:52:613.186 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: SANDESH: Send FAILED: 1789353472612975 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 15 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 96622 calls = 49 average_bytes = 1971 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 326 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 987 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 14 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 71 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 55 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1484, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 174, _iter59->first = sent _iter59->second = 69, ] ] ] 2026-09-14 Mon 02:37:52:613.214 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:37:53:652.290 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:53:665.615 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:53:665.978 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:53:665.991 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:37:56:588.815 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:35995 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:37:56:588.876 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:37:56:588.922 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:35995 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:37:56:589.034 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:56:589.044 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:37:56:589.045 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:37:56:589.133 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:37:56:589.305 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:56:589.316 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:37:56:589.372 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 35995 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:37:56:589.430 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:56:589.441 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:37:56:589.449 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:56:589.454 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:37:56:589.471 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:37:56:589.520 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:37:56:589.571 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:37:56:589.596 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:37:56:589.610 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:37:56:589.575 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:37:58:666.356 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:37:58:679.893 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:37:58:680.222 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:37:58:680.236 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:03:680.621 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:03:709.301 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:03:709.704 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:03:709.773 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:07:543.638 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:07:543.696 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:38:07:543.770 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: SANDESH: Send FAILED: 1789353487543484 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 251 dequeues = 251 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 313 dequeues = 313 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353472612896 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353476589569 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 6 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 8350 calls = 5 average_bytes = 1670 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:38:07:543.802 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:38:07:543.813 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:07:543.819 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:38:07:543.970 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: SANDESH: Send FAILED: 1789353487543587 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 15 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 104972 calls = 54 average_bytes = 1943 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 329 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 996 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 15 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 67 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 60 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1501, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 176, _iter59->first = sent _iter59->second = 75, ] ] ] 2026-09-14 Mon 02:38:07:543.990 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:38:08:710.158 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:08:723.963 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:08:724.371 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:08:724.386 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:13:724.732 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:13:738.638 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:13:738.983 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:13:738.999 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:18:739.397 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:18:753.788 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:18:754.198 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:18:754.282 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:22:613.049 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: 10.0.0.185:8086 EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:38:22:613.162 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:38:22:613.233 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: ClientInit : EvConnectTimerExpired 2026-09-14 Mon 02:38:22:613.415 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Return to idle with 1000 2026-09-14 Mon 02:38:22:613.426 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Idle 2026-09-14 Mon 02:38:22:613.529 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:38:22:613.553 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: SANDESH: Send FAILED: 1789353502613314 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:38:22:613.561 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:38:22:613.569 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvTcpDeleteSession in state Idle 2026-09-14 Mon 02:38:22:613.578 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:38:22:613.725 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: SANDESH: Send FAILED: 1789353502613511 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 15 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 104972 calls = 54 average_bytes = 1943 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 332 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1005 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 15 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 74 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 67 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 60 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1513, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 178, _iter59->first = sent _iter59->second = 75, ] ] ] 2026-09-14 Mon 02:38:22:613.750 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:38:23:613.593 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-09-14 Mon 02:38:23:613.952 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Connect : Start Connect timer 10.0.0.185:8086 2026-09-14 Mon 02:38:23:614.085 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:38:23:614.116 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: SANDESH: Send FAILED: 1789353503613696 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:38:23:614.126 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:38:23:614.134 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:38:23:614.335 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: SANDESH: Send FAILED: 1789353503614058 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 15 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 104972 calls = 54 average_bytes = 1943 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 332 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1005 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 15 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 75 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 67 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 60 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1513, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 180, _iter59->first = sent _iter59->second = 75, ] ] ] 2026-09-14 Mon 02:38:23:614.372 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:38:23:615.599 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:51580::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:38:23:615.637 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:51580::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:38:23:615.646 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:51580::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:38:23:616.869 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:38:23:616.902 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:38:23:616.942 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit 2026-09-14 Mon 02:38:23:616.969 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 16 2026-09-14 Mon 02:38:23:617.057 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:23:617.067 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:38:23:617.084 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: SANDESH: Send FAILED: 1789353503616934 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:38:23:617.091 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:38:23:617.098 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:23:617.104 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:38:23:617.126 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:23:617.132 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:38:23:617.279 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: SANDESH: Send FAILED: 1789353503617042 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 16 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 104972 calls = 54 average_bytes = 1943 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 332 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1005 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 52 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 15 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 76 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 60 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1514, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 182, _iter59->first = sent _iter59->second = 75, ] ] ] 2026-09-14 Mon 02:38:23:617.297 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:38:23:754.650 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:23:767.813 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:23:768.190 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:23:768.237 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:27:916.192 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:44269 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:38:27:916.281 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:38:27:916.333 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:44269 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:38:27:916.498 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:38:27:916.519 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:27:916.582 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:38:27:916.602 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:27:916.609 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:38:27:916.646 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:38:27:916.720 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 44269 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:38:27:918.386 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:27:918.414 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:38:27:918.432 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:27:918.442 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:38:27:918.452 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:27:918.461 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:38:27:918.553 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:38:27:918.623 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:38:27:918.655 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:38:27:918.727 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:38:28:768.598 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:28:782.261 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:28:782.619 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:28:782.648 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:33:783.091 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:33:796.286 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:33:796.643 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:33:796.655 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:37:543.923 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:37:543.982 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:38:37:544.057 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: SANDESH: Send FAILED: 1789353517543799 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 265 dequeues = 265 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 331 dequeues = 331 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353503616941 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353507918450 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 6 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 8350 calls = 3 average_bytes = 2783 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:38:37:544.084 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:38:37:544.095 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:37:544.101 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:38:37:544.242 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: SANDESH: Send FAILED: 1789353517543876 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 16 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 113322 calls = 57 average_bytes = 1988 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 335 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1014 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 16 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 77 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 69 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 65 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1531, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 184, _iter59->first = sent _iter59->second = 81, ] ] ] 2026-09-14 Mon 02:38:37:544.260 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:38:38:797.022 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:38:810.595 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:38:810.954 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:38:810.968 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:43:811.361 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:43:825.244 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:43:825.586 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:43:825.633 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:48:825.978 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:48:838.797 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:48:839.117 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:48:839.187 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:53:617.073 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: 10.0.0.185:8086 EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:38:53:617.194 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:38:53:617.266 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: ClientInit : EvConnectTimerExpired 2026-09-14 Mon 02:38:53:617.428 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Return to idle with 1000 2026-09-14 Mon 02:38:53:617.440 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Idle 2026-09-14 Mon 02:38:53:617.547 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:38:53:617.571 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: SANDESH: Send FAILED: 1789353533617315 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:38:53:617.579 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:38:53:617.588 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvTcpDeleteSession in state Idle 2026-09-14 Mon 02:38:53:617.633 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:38:53:617.806 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: SANDESH: Send FAILED: 1789353533617529 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 16 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 113322 calls = 57 average_bytes = 1988 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 338 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1023 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 16 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 79 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 69 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 65 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1543, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 186, _iter59->first = sent _iter59->second = 81, ] ] ] 2026-09-14 Mon 02:38:53:617.835 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:38:53:839.565 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:53:853.505 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:53:853.937 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:53:853.953 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:38:54:617.582 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-09-14 Mon 02:38:54:617.903 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Connect : Start Connect timer 10.0.0.185:8086 2026-09-14 Mon 02:38:54:618.020 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:38:54:618.049 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353534617670 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:38:54:618.056 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:38:54:618.064 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:38:54:618.228 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353534618002 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 16 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 113322 calls = 57 average_bytes = 1988 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 339 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1026 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 16 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 80 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 69 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 65 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1547, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 188, _iter59->first = sent _iter59->second = 81, ] ] ] 2026-09-14 Mon 02:38:54:618.516 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:38:54:620.078 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:35700::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:38:54:620.097 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:35700::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:38:54:620.105 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:35700::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:38:54:620.167 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:38:54:620.176 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:38:54:620.214 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit 2026-09-14 Mon 02:38:54:620.230 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 17 2026-09-14 Mon 02:38:54:620.318 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:54:620.337 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:38:54:620.364 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353534620199 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:38:54:620.378 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:38:54:620.386 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:54:620.392 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:38:54:620.404 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:54:620.410 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:38:54:620.562 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353534620301 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 17 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 113322 calls = 57 average_bytes = 1988 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 339 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1026 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 16 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 81 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 70 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 65 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1548, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 190, _iter59->first = sent _iter59->second = 81, ] ] ] 2026-09-14 Mon 02:38:54:620.583 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:38:58:704.963 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:34731 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:38:58:705.033 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:38:58:705.081 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:34731 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:38:58:705.211 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:58:705.226 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:38:58:705.231 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:38:58:705.270 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:38:58:705.370 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:58:705.381 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:38:58:705.399 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 34731 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:38:58:705.544 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:58:705.554 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:38:58:705.564 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:58:705.570 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:38:58:705.592 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:38:58:705.642 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:38:58:705.704 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:38:58:705.718 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:38:58:705.743 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:38:58:707.507 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:38:58:854.307 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:38:58:869.794 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:38:58:870.186 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:38:58:870.208 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:03:870.566 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:03:883.769 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:03:884.125 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:03:884.137 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:07:544.120 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:07:544.192 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:39:07:544.304 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353547543987 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 279 dequeues = 279 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 349 dequeues = 349 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353534620213 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353538705701 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 6 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 8350 calls = 5 average_bytes = 1670 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:39:07:544.334 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:39:07:544.346 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:07:544.352 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:39:07:544.486 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353547544074 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 17 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 121672 calls = 62 average_bytes = 1962 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 341 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1032 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 17 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 82 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 71 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 70 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1561, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 192, _iter59->first = sent _iter59->second = 87, ] ] ] 2026-09-14 Mon 02:39:07:544.516 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:39:08:884.772 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:08:897.742 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:08:898.128 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:08:898.142 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:13:898.488 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:13:913.712 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:13:914.194 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:13:914.247 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:18:914.639 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:18:927.708 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:18:928.087 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:18:928.098 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:23:928.488 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:23:941.833 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:23:942.148 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:23:942.163 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:24:620.349 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: 10.0.0.185:8086 EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:39:24:621.473 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:39:24:621.485 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: ClientInit : EvConnectTimerExpired 2026-09-14 Mon 02:39:24:621.637 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Return to idle with 1000 2026-09-14 Mon 02:39:24:621.648 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Idle 2026-09-14 Mon 02:39:24:621.767 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:39:24:621.795 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: SANDESH: Send FAILED: 1789353564621524 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:39:24:621.805 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:39:24:621.815 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvTcpDeleteSession in state Idle 2026-09-14 Mon 02:39:24:621.860 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:39:24:622.006 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: SANDESH: Send FAILED: 1789353564621745 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 17 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 121672 calls = 62 average_bytes = 1962 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 345 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1044 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 17 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 84 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 71 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 70 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1577, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 194, _iter59->first = sent _iter59->second = 87, ] ] ] 2026-09-14 Mon 02:39:24:622.031 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:39:25:621.810 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-09-14 Mon 02:39:25:622.936 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Connect : Start Connect timer 10.0.0.185:8086 2026-09-14 Mon 02:39:25:623.108 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:39:25:623.138 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: SANDESH: Send FAILED: 1789353565621910 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:39:25:623.148 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:39:25:623.179 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:39:25:623.352 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: SANDESH: Send FAILED: 1789353565623069 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 17 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 121672 calls = 62 average_bytes = 1962 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 345 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1044 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 57 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 17 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 85 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 71 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 70 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1577, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 196, _iter59->first = sent _iter59->second = 87, ] ] ] 2026-09-14 Mon 02:39:25:623.386 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:39:25:623.715 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:50494::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:39:25:623.741 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:50494::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:39:25:623.751 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:50494::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:39:25:623.816 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:39:25:623.830 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:39:25:623.866 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: ClientInit 2026-09-14 Mon 02:39:25:623.885 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 18 2026-09-14 Mon 02:39:25:623.962 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:25:623.982 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:39:25:624.001 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: SANDESH: Send FAILED: 1789353565623859 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:39:25:624.041 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:39:25:624.050 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:25:624.056 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:39:25:624.067 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:25:624.072 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:39:25:624.200 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: SANDESH: Send FAILED: 1789353565623946 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 18 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 121672 calls = 62 average_bytes = 1962 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 345 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1044 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 58 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 17 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 86 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 72 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 70 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1578, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 198, _iter59->first = sent _iter59->second = 87, ] ] ] 2026-09-14 Mon 02:39:25:624.234 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:39:26:251.125 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:34561 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:39:26:251.209 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:39:26:251.253 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:34561 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:39:26:251.445 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:26:251.453 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:39:26:251.460 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:39:26:251.552 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:39:26:252.491 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:26:252.519 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:39:26:252.549 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 34561 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:39:26:252.655 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:26:252.671 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:39:26:252.681 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:26:252.688 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:39:26:252.708 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:39:26:252.763 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:39:26:252.797 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:39:26:252.819 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:26:252.844 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:39:26:252.882 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:39:28:942.528 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:28:955.668 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:28:956.074 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:28:956.089 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:33:956.442 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:33:970.059 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:33:970.459 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:33:970.474 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:37:544.474 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:37:544.539 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:39:37:544.638 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: SANDESH: Send FAILED: 1789353577544329 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 293 dequeues = 293 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 367 dequeues = 367 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353565623865 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353566252813 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 6 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 8350 calls = 5 average_bytes = 1670 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:39:37:544.668 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:39:37:544.680 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:37:544.687 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:39:37:544.842 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: SANDESH: Send FAILED: 1789353577544415 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 18 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 130022 calls = 67 average_bytes = 1940 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 347 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1050 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 59 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 18 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 87 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 73 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 75 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1591, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 200, _iter59->first = sent _iter59->second = 93, ] ] ] 2026-09-14 Mon 02:39:37:544.866 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:39:38:970.830 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:38:987.776 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:39:007.999 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:39:008.053 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:44:008.796 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:44:022.508 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:44:022.904 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:44:022.947 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:49:023.374 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:49:036.586 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:49:036.988 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:49:037.023 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:54:037.433 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:54:051.873 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:54:052.298 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:54:052.324 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:39:54:340.667 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:38309 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:39:54:340.730 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:39:54:340.774 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:38309 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:39:54:340.931 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552799446592, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:39:54:340.977 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552799446592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:54:340.986 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552799446592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:39:54:340.998 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552799446592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:54:341.003 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552799446592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:39:54:341.052 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:39:54:341.104 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 38309 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:39:54:341.333 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:54:341.346 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:39:54:341.367 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:54:341.373 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:39:54:341.405 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:39:54:341.442 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:39:54:341.529 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:39:54:341.560 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:54:341.592 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:39:54:341.628 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:39:55:623.952 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: 10.0.0.185:8086 EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:39:55:624.048 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:39:55:624.058 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: ClientInit : EvConnectTimerExpired 2026-09-14 Mon 02:39:55:624.247 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Return to idle with 1000 2026-09-14 Mon 02:39:55:624.259 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Idle 2026-09-14 Mon 02:39:55:624.364 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:39:55:624.388 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: SANDESH: Send FAILED: 1789353595624125 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:39:55:624.397 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:39:55:624.420 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Processing scm::EvTcpDeleteSession in state Idle 2026-09-14 Mon 02:39:55:624.428 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:39:55:624.575 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: SANDESH: Send FAILED: 1789353595624347 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 18 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 136760 calls = 70 average_bytes = 1953 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 351 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1062 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 59 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 18 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 89 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 80 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1612, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 202, _iter59->first = sent _iter59->second = 98, ] ] ] 2026-09-14 Mon 02:39:55:624.618 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:39:56:624.368 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-09-14 Mon 02:39:56:624.776 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Connect : Start Connect timer 10.0.0.185:8086 2026-09-14 Mon 02:39:56:624.891 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:39:56:624.919 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: SANDESH: Send FAILED: 1789353596624469 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:39:56:624.940 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:39:56:624.949 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:39:56:625.183 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: SANDESH: Send FAILED: 1789353596624872 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 18 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 136760 calls = 70 average_bytes = 1953 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 351 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1062 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 18 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 90 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 80 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1612, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 204, _iter59->first = sent _iter59->second = 98, ] ] ] 2026-09-14 Mon 02:39:56:625.220 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:39:56:626.309 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:57128::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:39:56:626.355 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:57128::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:39:56:626.375 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:57128::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:39:56:627.596 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:39:56:627.632 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:39:56:627.669 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: ClientInit 2026-09-14 Mon 02:39:56:627.697 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 19 2026-09-14 Mon 02:39:56:627.792 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:56:627.801 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:39:56:627.837 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: SANDESH: Send FAILED: 1789353596627662 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:39:56:627.844 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:39:56:627.853 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:56:627.860 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:39:56:627.871 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:39:56:627.877 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:39:56:628.067 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: SANDESH: Send FAILED: 1789353596627774 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 19 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 136760 calls = 70 average_bytes = 1953 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 351 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1062 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 18 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 91 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 80 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1613, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 206, _iter59->first = sent _iter59->second = 98, ] ] ] 2026-09-14 Mon 02:39:56:628.093 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:39:59:052.687 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:39:59:066.986 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:39:59:067.464 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:39:59:067.526 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:04:067.897 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:04:081.867 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:04:082.372 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:04:082.389 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:07:544.655 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:07:544.706 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:40:07:544.799 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: SANDESH: Send FAILED: 1789353607544525 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 307 dequeues = 307 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 385 dequeues = 385 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353596627668 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353596627870 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 1 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 1612 calls = 1 average_bytes = 1612 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:40:07:544.811 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:40:07:544.822 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:07:544.829 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:40:07:545.003 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: SANDESH: Send FAILED: 1789353607544611 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 19 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 138372 calls = 71 average_bytes = 1948 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 353 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1068 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 19 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 92 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 80 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1621, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 208, _iter59->first = sent _iter59->second = 99, ] ] ] 2026-09-14 Mon 02:40:07:545.025 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:40:09:082.763 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:09:096.360 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:09:096.848 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:09:096.931 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:14:097.338 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:14:110.642 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:14:111.015 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:14:111.032 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:19:111.377 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:19:125.214 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:19:125.583 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:19:125.647 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:19:156.182 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8092::10.20.0.17:54282 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:40:20:477.251 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8092::10.20.0.17:58000 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:40:24:126.110 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:24:139.451 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:24:139.801 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:24:139.833 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:25:396.833 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:38081 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:40:25:396.897 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:40:25:397.234 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:38081 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:40:25:397.436 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:40:25:397.822 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:25:397.929 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:40:25:397.947 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:25:397.954 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:40:25:397.982 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:40:25:398.098 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 38081 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:40:25:399.878 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:25:399.912 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:40:25:399.935 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:25:399.945 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:40:25:399.953 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:25:399.958 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:40:25:399.936 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:40:25:400.013 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:40:25:400.087 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:40:25:400.121 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552749065792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:40:26:627.842 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: 10.0.0.185:8086 EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:40:26:627.957 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:40:26:628.004 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvConnectTimerExpired 2026-09-14 Mon 02:40:26:628.135 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Return to idle with 1000 2026-09-14 Mon 02:40:26:628.144 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Idle 2026-09-14 Mon 02:40:26:628.258 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:40:26:628.281 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353626628048 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:40:26:628.298 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:40:26:628.307 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvTcpDeleteSession in state Idle 2026-09-14 Mon 02:40:26:628.315 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:40:26:628.465 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353626628242 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 19 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 145110 calls = 73 average_bytes = 1987 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 357 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1080 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 19 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 94 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 85 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1644, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 210, _iter59->first = sent _iter59->second = 104, ] ] ] 2026-09-14 Mon 02:40:26:628.483 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:40:27:628.363 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-09-14 Mon 02:40:27:628.703 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Connect : Start Connect timer 10.0.0.185:8086 2026-09-14 Mon 02:40:27:628.853 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:40:27:628.885 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: SANDESH: Send FAILED: 1789353627628474 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:40:27:628.893 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:40:27:628.903 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:40:27:629.076 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: SANDESH: Send FAILED: 1789353627628831 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 19 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 145110 calls = 73 average_bytes = 1987 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 357 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1080 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 63 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 19 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 95 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 85 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1644, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 212, _iter59->first = sent _iter59->second = 104, ] ] ] 2026-09-14 Mon 02:40:27:629.528 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:40:27:631.222 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:60210::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:40:27:631.242 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:60210::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:40:27:631.250 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:60210::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:40:27:631.329 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:40:27:631.340 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:40:27:631.371 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit 2026-09-14 Mon 02:40:27:631.385 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 20 2026-09-14 Mon 02:40:27:631.471 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:27:631.479 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:40:27:631.495 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: SANDESH: Send FAILED: 1789353627631364 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:40:27:631.502 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:40:27:631.509 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:27:631.515 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:40:27:631.525 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:27:631.530 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:40:27:631.665 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: SANDESH: Send FAILED: 1789353627631456 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 20 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 145110 calls = 73 average_bytes = 1987 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 357 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1080 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 19 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 96 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 85 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1645, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 214, _iter59->first = sent _iter59->second = 104, ] ] ] 2026-09-14 Mon 02:40:27:631.689 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:40:29:140.219 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:29:163.367 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:29:163.748 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:29:163.768 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:34:164.123 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:34:177.975 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:34:178.364 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:34:178.391 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:37:544.971 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:37:545.028 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:40:37:545.121 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: SANDESH: Send FAILED: 1789353637544825 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 321 dequeues = 321 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 403 dequeues = 403 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353627631370 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353627631524 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 1 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 1612 calls = 1 average_bytes = 1612 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:40:37:545.135 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:40:37:545.145 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:37:545.152 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:40:37:545.312 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: SANDESH: Send FAILED: 1789353637544907 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 20 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 146722 calls = 74 average_bytes = 1982 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 359 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1086 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 20 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 97 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 85 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1653, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 216, _iter59->first = sent _iter59->second = 105, ] ] ] 2026-09-14 Mon 02:40:37:545.337 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:40:39:178.695 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:39:192.795 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:39:193.288 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:39:193.328 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:44:193.698 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:44:206.788 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:44:207.232 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:44:207.278 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:49:207.630 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:49:221.426 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:49:222.028 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:49:222.071 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:52:411.398 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:36961 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:40:52:411.461 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:40:52:411.506 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:36961 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:40:52:411.708 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:52:411.724 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:40:52:411.724 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552786851392, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:40:52:411.783 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:40:52:411.885 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:52:411.898 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:40:52:411.930 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 36961 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:40:52:412.197 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:52:412.222 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:40:52:412.249 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:52:412.256 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:40:52:412.278 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:40:52:412.317 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:40:52:412.359 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:52:412.366 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:40:52:412.417 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:40:52:412.422 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:40:54:222.350 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:54:235.766 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:54:239.981 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:54:240.029 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:40:57:632.686 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: 10.0.0.185:8086 EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:40:57:632.791 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:40:57:632.825 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: ClientInit : EvConnectTimerExpired 2026-09-14 Mon 02:40:57:632.977 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Return to idle with 1000 2026-09-14 Mon 02:40:57:632.986 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Idle 2026-09-14 Mon 02:40:57:633.144 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:40:57:633.173 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: SANDESH: Send FAILED: 1789353657632868 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:40:57:633.201 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:40:57:633.226 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Processing scm::EvTcpDeleteSession in state Idle 2026-09-14 Mon 02:40:57:633.239 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:40:57:633.390 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: SANDESH: Send FAILED: 1789353657633112 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 20 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 153460 calls = 78 average_bytes = 1967 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 363 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1098 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 20 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 99 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 80 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 90 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1674, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 218, _iter59->first = sent _iter59->second = 110, ] ] ] 2026-09-14 Mon 02:40:57:633.411 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:40:58:633.157 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-09-14 Mon 02:40:58:633.530 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Connect : Start Connect timer 10.0.0.185:8086 2026-09-14 Mon 02:40:58:633.651 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:40:58:633.675 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: SANDESH: Send FAILED: 1789353658633277 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:40:58:633.683 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:40:58:633.692 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:40:58:633.866 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: SANDESH: Send FAILED: 1789353658633630 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 20 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 153460 calls = 78 average_bytes = 1967 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 363 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1098 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 66 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 20 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 80 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 90 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1674, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 220, _iter59->first = sent _iter59->second = 110, ] ] ] 2026-09-14 Mon 02:40:58:633.895 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:40:58:635.184 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:53140::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:40:58:635.236 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:53140::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:40:58:635.246 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:53140::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:40:58:635.360 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:40:58:635.394 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:40:58:635.447 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: ClientInit 2026-09-14 Mon 02:40:58:635.469 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 21 2026-09-14 Mon 02:40:58:635.571 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:58:635.580 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:40:58:635.599 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: SANDESH: Send FAILED: 1789353658635434 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:40:58:635.606 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:40:58:635.614 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:58:635.620 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:40:58:635.630 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:40:58:635.635 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:40:58:635.793 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: SANDESH: Send FAILED: 1789353658635556 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 21 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 153460 calls = 78 average_bytes = 1967 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 363 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1098 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 67 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 20 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 101 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 90 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1675, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 222, _iter59->first = sent _iter59->second = 110, ] ] ] 2026-09-14 Mon 02:40:58:636.154 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:40:59:240.448 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:40:59:254.947 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:40:59:255.351 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:40:59:255.387 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:04:255.757 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:04:269.300 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:04:269.734 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:04:269.774 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:07:545.183 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:07:545.278 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:41:07:545.369 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: SANDESH: Send FAILED: 1789353667545020 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 335 dequeues = 335 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 421 dequeues = 421 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353658635445 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353658635629 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 1 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 1612 calls = 1 average_bytes = 1612 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:41:07:545.383 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:41:07:545.395 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:07:545.405 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:41:07:545.567 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: SANDESH: Send FAILED: 1789353667545109 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 21 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 155072 calls = 79 average_bytes = 1962 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 365 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1104 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 21 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 102 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 90 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1683, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 224, _iter59->first = sent _iter59->second = 111, ] ] ] 2026-09-14 Mon 02:41:07:545.606 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:41:09:270.142 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:09:283.800 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:09:284.168 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:09:284.225 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:14:284.530 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:14:298.492 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:14:298.981 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:14:299.027 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:19:299.422 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:19:313.643 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:19:314.197 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:19:314.249 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:23:469.137 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:33063 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:41:23:469.529 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:41:23:469.595 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:33063 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:41:23:469.728 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:23:469.768 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:41:23:469.736 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:41:23:469.810 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:41:23:469.923 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:23:469.946 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:41:23:469.988 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 33063 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:41:23:470.057 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:23:470.080 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:41:23:470.088 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:23:470.093 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:41:23:470.099 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:23:470.105 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:41:23:470.137 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:41:23:470.182 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552807843392, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:41:23:470.218 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:41:23:470.269 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:41:24:314.611 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:24:328.605 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:24:329.070 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:24:329.084 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:28:635.636 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552757462592, Pid 64205]: 10.0.0.185:8086 EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:41:28:635.747 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:41:28:635.792 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: ClientInit : EvConnectTimerExpired 2026-09-14 Mon 02:41:28:635.941 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Return to idle with 1000 2026-09-14 Mon 02:41:28:635.957 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Idle 2026-09-14 Mon 02:41:28:636.076 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:41:28:636.105 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: SANDESH: Send FAILED: 1789353688635849 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:41:28:636.116 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:41:28:636.127 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvTcpDeleteSession in state Idle 2026-09-14 Mon 02:41:28:636.138 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:41:28:636.316 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: SANDESH: Send FAILED: 1789353688636059 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 21 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 161810 calls = 82 average_bytes = 1973 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 369 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1116 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 21 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 104 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 95 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1704, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 226, _iter59->first = sent _iter59->second = 116, ] ] ] 2026-09-14 Mon 02:41:28:636.347 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:41:29:329.420 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:29:342.684 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:29:343.134 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:29:343.148 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:29:636.183 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-09-14 Mon 02:41:29:636.517 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Connect : Start Connect timer 10.0.0.185:8086 2026-09-14 Mon 02:41:29:636.632 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:41:29:636.660 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: SANDESH: Send FAILED: 1789353689636297 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:41:29:636.670 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:41:29:636.679 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:41:29:636.882 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: SANDESH: Send FAILED: 1789353689636615 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 21 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 161810 calls = 82 average_bytes = 1973 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 370 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1119 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 69 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 21 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 105 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 95 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1708, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 228, _iter59->first = sent _iter59->second = 116, ] ] ] 2026-09-14 Mon 02:41:29:636.912 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:41:29:638.952 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:46040::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:41:29:638.973 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:46040::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:41:29:638.982 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:46040::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:41:29:639.081 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:41:29:639.092 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:41:29:639.123 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit 2026-09-14 Mon 02:41:29:639.141 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 22 2026-09-14 Mon 02:41:29:639.263 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:29:639.273 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:41:29:639.289 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: SANDESH: Send FAILED: 1789353689639117 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:41:29:639.304 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:41:29:639.312 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:29:639.317 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:41:29:639.333 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:29:639.338 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:41:29:639.469 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: SANDESH: Send FAILED: 1789353689639251 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 22 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 161810 calls = 82 average_bytes = 1973 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 370 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1119 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 21 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 106 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 83 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 95 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1709, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 230, _iter59->first = sent _iter59->second = 116, ] ] ] 2026-09-14 Mon 02:41:29:639.485 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552744867392, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:41:34:343.564 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:34:357.441 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:34:357.807 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:34:357.821 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:37:545.486 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:37:545.549 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:41:37:545.636 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: SANDESH: Send FAILED: 1789353697545336 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 349 dequeues = 349 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 439 dequeues = 439 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353689639123 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353689639332 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 1 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 1612 calls = 1 average_bytes = 1612 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:41:37:545.649 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:41:37:545.660 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:37:545.667 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:41:37:545.811 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: SANDESH: Send FAILED: 1789353697545423 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 22 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 163422 calls = 83 average_bytes = 1968 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 371 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1122 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 71 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 22 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 107 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 83 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 95 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1713, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 232, _iter59->first = sent _iter59->second = 117, ] ] ] 2026-09-14 Mon 02:41:37:545.840 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:41:39:358.180 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:39:372.567 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:39:420.935 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:39:420.983 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:44:421.364 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:44:434.698 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:44:435.040 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:44:435.069 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:49:435.445 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:49:449.327 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:49:449.700 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:49:449.740 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:50:478.310 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:34853 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:41:50:478.435 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:41:50:478.508 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:34853 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:41:50:478.674 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:41:50:478.718 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:50:478.848 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:41:50:478.883 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:50:478.892 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:41:50:478.958 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:41:50:479.071 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 34853 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:41:50:481.426 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:50:481.467 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:41:50:481.493 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:50:481.501 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:41:50:481.528 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:41:50:481.630 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552774256192, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:41:50:481.699 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:41:50:481.705 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:41:50:481.816 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552761660992, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:41:50:481.856 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:41:54:450.107 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:54:463.539 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:54:463.875 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:54:463.889 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:59:464.246 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:41:59:480.758 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:41:59:481.182 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:41:59:481.225 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:41:59:639.322 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: 10.0.0.185:8086 EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:41:59:639.429 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-09-14 Mon 02:41:59:639.469 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: ClientInit : EvConnectTimerExpired 2026-09-14 Mon 02:41:59:639.659 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Return to idle with 1000 2026-09-14 Mon 02:41:59:639.675 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Idle 2026-09-14 Mon 02:41:59:639.791 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:41:59:639.818 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: SANDESH: Send FAILED: 1789353719639509 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:41:59:639.827 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:41:59:639.835 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvTcpDeleteSession in state Idle 2026-09-14 Mon 02:41:59:639.878 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Processing scm::EvSandeshSend in state Idle 2026-09-14 Mon 02:41:59:640.050 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: SANDESH: Send FAILED: 1789353719639773 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 22 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 170160 calls = 86 average_bytes = 1978 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 376 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1137 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 71 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 22 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 109 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 100 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1738, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 234, _iter59->first = sent _iter59->second = 122, ] ] ] 2026-09-14 Mon 02:41:59:640.085 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552803644992, Pid 64205]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-09-14 Mon 02:42:00:639.863 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-09-14 Mon 02:42:00:641.824 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.185:42778::10.0.0.185:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-09-14 Mon 02:42:00:641.848 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:42778::10.0.0.185:8086(-1) OnSessionEvent TCP Connected 2026-09-14 Mon 02:42:00:641.856 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: 10.0.0.185:42778::10.0.0.185:8086(-1):EvTcpConnected 2026-09-14 Mon 02:42:00:641.890 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Connect : Start Connect timer 10.0.0.185:8086 2026-09-14 Mon 02:42:00:641.992 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state Connect 2026-09-14 Mon 02:42:00:642.015 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: SANDESH: Send FAILED: 1789353720639968 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:42:00:642.023 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-09-14 Mon 02:42:00:642.032 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvTcpConnected in state Connect 2026-09-14 Mon 02:42:00:642.038 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Connect : EvTcpConnected : Cancelling Connect timer 2026-09-14 Mon 02:42:00:642.075 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit 2026-09-14 Mon 02:42:00:642.098 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6276-1:contrail-dns:0:Control count 23 2026-09-14 Mon 02:42:00:642.150 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:42:00:642.159 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:42:00:642.346 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: SANDESH: Send FAILED: 1789353720641974 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 22 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 170160 calls = 86 average_bytes = 1978 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 376 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1137 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 22 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 110 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 85 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 100 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1739, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 236, _iter59->first = sent _iter59->second = 122, ] ] ] 2026-09-14 Mon 02:42:00:642.372 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:42:00:642.385 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:42:00:642.391 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-09-14 Mon 02:42:00:642.406 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: SANDESH: Send FAILED: 1789353720642071 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.185:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.185:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.185:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-09-14 Mon 02:42:00:642.770 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Received UVE message in wrong state : NodeStatusUVE 2026-09-14 Mon 02:42:00:642.787 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:42:00:642.794 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-09-14 Mon 02:42:00:642.807 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:42:00:642.812 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:42:00:642.946 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: SANDESH: Send FAILED: 1789353720642139 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 23 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 170160 calls = 86 average_bytes = 1978 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 376 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1137 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 73 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 22 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 110 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 85 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 100 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1739, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 237, _iter59->first = sent _iter59->second = 122, ] ] ] 2026-09-14 Mon 02:42:00:642.967 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552267888192, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:42:04:481.642 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:42:04:504.927 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:42:04:508.958 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:42:04:508.980 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:42:07:545.710 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:42:07:545.767 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:42:07:545.849 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353727545573 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 363 dequeues = 363 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 457 dequeues = 457 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1789353720642074 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1789353720642806 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 1 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 1612 calls = 1 average_bytes = 1612 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 4 ] 2026-09-14 Mon 02:42:07:545.863 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:42:07:545.873 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:42:07:545.879 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-09-14 Mon 02:42:07:546.038 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: SANDESH: Send FAILED: 1789353727545662 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6276-1:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 23 pid = 64205 http_port = 8092 start_time = 1789351837485484 collector_name = collector_ip = 10.0.0.185:8086 collector_list= [ [ (*_iter40) = 10.0.0.185:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 171772 calls = 87 average_bytes = 1974 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 377 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1140 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 74 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 23 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 112 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 85 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 100 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppDeleteConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1743, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 240, _iter59->first = sent _iter59->second = 123, ] ] ] 2026-09-14 Mon 02:42:07:546.056 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552778454592, Pid 64205]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-09-14 Mon 02:42:09:509.420 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:42:09:522.765 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:42:09:523.135 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:42:09:523.209 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:42:14:523.527 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:42:14:536.281 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:42:14:536.564 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:42:14:536.580 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:42:19:539.408 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:42:19:553.115 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:42:19:553.495 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:42:19:553.524 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2026-09-14 Mon 02:42:20:189.650 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 SEND Xmpp creating dynamic channel 10.20.0.17:8093::10.20.0.17:34571 controller/src/xmpp/xmpp_server.cc 577 2026-09-14 Mon 02:42:20:189.711 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : controller/src/xmpp/xmpp_connection.cc 725 2026-09-14 Mon 02:42:20:189.760 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552816539136, Pid 64205]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8093::10.20.0.17:34571 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-09-14 Mon 02:42:20:189.866 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.17 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2026-09-14 Mon 02:42:20:189.871 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:42:20:189.971 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:42:20:189.991 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:42:20:189.999 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552753264192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:42:20:190.035 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:42:20:190.127 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552770057792, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 34571 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:42:20:191.272 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:42:20:191.292 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:42:20:191.309 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:42:20:191.314 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552782652992, Pid 64205]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-6276-1:10.20.0.17 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-6276-1 To : cn-jenkins-deploy-platform-ansible-os-6276-1/dns controller/src/xmpp/xmpp_connection.cc 732 2026-09-14 Mon 02:42:20:191.318 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:42:20:191.362 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: Processing scm::EvSandeshSend in state ClientInit 2026-09-14 Mon 02:42:20:191.370 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552795248192, Pid 64205]: ClientInit : EvSandeshSend : XmppConnectionLog 2026-09-14 Mon 02:42:20:191.394 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-ansible-os-6276-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:42:20:191.454 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Established event = EvStop ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:42:20:191.488 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552765859392, Pid 64205]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1711 2026-09-14 Mon 02:42:24:553.840 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-09-14 Mon 02:42:24:567.101 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-09-14 Mon 02:42:24:567.482 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 98 2026-09-14 Mon 02:42:24:567.507 UTC cn-jenkins-deploy-platform-ansible-os-6276-1 [Thread 140552791049792, Pid 64205]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122