2026-06-08 Mon 05:45:10:659.567 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: http-server-port 8091 2026-06-08 Mon 05:45:10:659.659 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Max-tasks 16 2026-06-08 Mon 05:45:10:659.667 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Max-slice 100 2026-06-08 Mon 05:45:10:659.673 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Collectors 10.0.0.38:8086 2026-06-08 Mon 05:45:10:659.678 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Collectors 10.0.0.50:8086 2026-06-08 Mon 05:45:10:659.683 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Collectors 10.0.0.33:8086 2026-06-08 Mon 05:45:10:659.714 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: ROLE : Generator 2026-06-08 Mon 05:45:10:659.720 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: MODULE : contrail-query-engine 2026-06-08 Mon 05:45:10:659.726 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-5902-1. 2026-06-08 Mon 05:45:10:659.731 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: NODE TYPE : Database 2026-06-08 Mon 05:45:10:660.383 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: INSTANCE ID : 0 2026-06-08 Mon 05:45:10:660.400 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: HTTP SERVER PORT : 8091 2026-06-08 Mon 05:45:10:660.408 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-06-08 Mon 05:45:10:663.663 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: HTTP Introspect Init 2026-06-08 Mon 05:45:10:663.774 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8091 Initialization complete src/contrail-common/io/tcp_server.cc 137 2026-06-08 Mon 05:45:10:663.849 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Sandesh Http Server Port: 8091 2026-06-08 Mon 05:45:10:663.922 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: NOT Writing http_port 8091TO : /tmp/contrail-query-engine.0.http_port 2026-06-08 Mon 05:45:10:663.929 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: CONNECT TO COLLECTOR: 1 2026-06-08 Mon 05:45:10:667.492 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Idle 2026-06-08 Mon 05:45:10:667.643 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: No Client: 1780897510667584 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:10:667.776 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: Logging: DISABLED -> ENABLED 2026-06-08 Mon 05:45:10:667.791 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-06-08 Mon 05:45:10:667.801 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Cassandra Servers: 10.0.0.38:9042 10.0.0.50:9042 10.0.0.33:9042 2026-06-08 Mon 05:45:10:672.188 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Processing scm::EvStart in state Idle 2026-06-08 Mon 05:45:10:672.287 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Disconnect 2026-06-08 Mon 05:45:10:672.370 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Processing scm::EvCollectorUpdate in state Disconnect 2026-06-08 Mon 05:45:10:672.591 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-08 Mon 05:45:10:672.668 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:45:10:672.691 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: SANDESH: Send FAILED: 1780897510672278 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-08 Mon 05:45:10:672.701 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:45:10:672.711 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:45:10:672.805 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: SANDESH: Send FAILED: 1780897510672355 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:10:672.829 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:10:672.839 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:45:10:672.906 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: SANDESH: Send FAILED: 1780897510672405 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:10:672.921 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:10:672.931 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:45:10:672.945 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: SANDESH: Send FAILED: 1780897510672424 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-08 Mon 05:45:10:672.952 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:45:10:672.960 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:45:10:673.020 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: SANDESH: Send FAILED: 1780897510672645 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:10:673.034 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463898486336, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:10:674.269 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-08 Mon 05:45:10:674.289 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-08 Mon 05:45:10:675.473 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 0 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-08 Mon 05:45:10:676.214 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:45:10:676.246 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780897510674316 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-08 Mon 05:45:10:676.253 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:45:15:678.035 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-08 Mon 05:45:15:678.087 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-08 Mon 05:45:15:678.118 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 1 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-08 Mon 05:45:20:680.480 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-08 Mon 05:45:20:680.517 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-08 Mon 05:45:20:680.533 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 2 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-08 Mon 05:45:25:683.047 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-08 Mon 05:45:25:683.091 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-08 Mon 05:45:25:683.107 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 3 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-08 Mon 05:45:30:751.273 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1003 2026-06-08 Mon 05:45:30:751.327 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-08 Mon 05:45:30:751.352 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 4 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-08 Mon 05:45:35:794.506 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1003 2026-06-08 Mon 05:45:35:794.554 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-08 Mon 05:45:35:794.577 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 5 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-08 Mon 05:45:40:833.445 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1003 2026-06-08 Mon 05:45:40:833.510 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-08 Mon 05:45:40:833.534 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 6 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-08 Mon 05:45:45:923.549 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1003 2026-06-08 Mon 05:45:45:923.633 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-08 Mon 05:45:45:923.650 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 7 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-08 Mon 05:45:50:992.748 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-08 Mon 05:45:50:993.919 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 8 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-08 Mon 05:45:56:029.028 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-06-08 Mon 05:46:01:034.159 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x238b1c0 2026-06-08 Mon 05:46:01:034.215 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.234 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.247 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2390180 2026-06-08 Mon 05:46:01:034.253 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.261 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.271 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x23908e0 2026-06-08 Mon 05:46:01:034.282 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.272 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:46:01:034.292 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.352 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2391200 2026-06-08 Mon 05:46:01:034.358 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.361 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: SANDESH: Send FAILED: 1780897561033980 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-08 Mon 05:46:01:034.393 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:46:01:034.369 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.418 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2391d20 2026-06-08 Mon 05:46:01:034.423 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.430 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.457 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:46:01:034.464 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:46:01:034.479 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2395fa0 2026-06-08 Mon 05:46:01:034.485 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.504 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.517 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2396a80 2026-06-08 Mon 05:46:01:034.523 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.530 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.540 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2397560 2026-06-08 Mon 05:46:01:034.545 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.552 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.560 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2398040 2026-06-08 Mon 05:46:01:034.565 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.573 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.581 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2398b00 2026-06-08 Mon 05:46:01:034.586 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.593 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.604 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2393f00 2026-06-08 Mon 05:46:01:034.609 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.625 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.635 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2392820 2026-06-08 Mon 05:46:01:034.639 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.647 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.655 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2393340 2026-06-08 Mon 05:46:01:034.660 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.667 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.693 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x23949e0 2026-06-08 Mon 05:46:01:034.699 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.706 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.716 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x23954c0 2026-06-08 Mon 05:46:01:034.721 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:46:01:034.732 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:46:01:034.506 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:46:01:034.774 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:46:01:034.944 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:46:01:034.994 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.016 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.001 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Idle 2026-06-08 Mon 05:46:01:035.035 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.115 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.146 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:46:01:035.149 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.186 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.231 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.253 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.265 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SANDESH: Send FAILED: 1780897561034859 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 43, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:46:01:035.295 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:46:01:035.268 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.312 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:46:01:035.351 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SANDESH: Send FAILED: 1780897561034918 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-08 Mon 05:46:01:035.360 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:46:01:035.328 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.373 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:46:01:035.427 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.448 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.455 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.473 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.484 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.492 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:46:01:035.512 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.520 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.530 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.536 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.545 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.551 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.561 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.567 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.576 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.582 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.591 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.597 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.606 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.609 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SANDESH: Send FAILED: 1780897561035126 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 45 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 47, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:46:01:035.613 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.637 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:46:01:035.651 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.672 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.665 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:46:01:035.693 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.700 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.704 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SANDESH: Send FAILED: 1780897561035222 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:46:01:035.710 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:46:01:035.728 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:46:01:035.718 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:46:01:035.744 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.754 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:46:01:035.784 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SANDESH: Send FAILED: 1780897561035248 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:46:01:035.787 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.801 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:46:01:035.821 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.826 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:46:01:035.835 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.857 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.857 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SANDESH: Send FAILED: 1780897561035779 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:46:01:035.880 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:46:01:035.902 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:46:01:035.868 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.921 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.921 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SANDESH: Send FAILED: 1780897561035817 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:46:01:035.946 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:46:01:035.932 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:035.989 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:036.004 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:46:01:036.062 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:46:01:036.108 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SANDESH: Send FAILED: 1780897561035994 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780897561035000 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780897561035900 ] max_sm_queue_count = 5 ] 2026-06-08 Mon 05:46:01:036.117 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:46:01:036.126 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:46:01:036.228 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SANDESH: Send FAILED: 1780897561036044 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 87 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 89, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 14, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:46:01:036.247 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:46:02:035.158 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463894287936, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:46:02:035.436 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463894287936, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-08 Mon 05:46:02:035.577 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463894287936, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:46:02:035.616 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463894287936, Pid 1]: SANDESH: Send FAILED: 1780897562035237 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:46:02:035.640 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463894287936, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:46:02:035.652 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463894287936, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:46:02:035.732 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463894287936, Pid 1]: SANDESH: Send FAILED: 1780897562035559 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 87 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 89, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 16, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:46:02:035.748 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463894287936, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:46:02:038.219 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:43004::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:46:02:039.197 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:43004::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:46:02:039.218 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:43004::10.0.0.33:8086(-1):EvTcpConnected 2026-06-08 Mon 05:46:02:039.727 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:46:02:039.749 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:46:02:039.806 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: ClientInit 2026-06-08 Mon 05:46:02:039.827 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-1.:contrail-query-engine:0:Database count 1 2026-06-08 Mon 05:46:02:039.954 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:46:02:039.979 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:46:02:040.008 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: SANDESH: Send FAILED: 1780897562039797 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:46:02:040.027 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:46:02:040.038 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:46:02:040.046 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:46:02:040.073 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:46:02:040.081 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:46:02:040.199 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: SANDESH: Send FAILED: 1780897562039930 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 87 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:46:02:040.221 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:46:02:048.398 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: OnMessage control in state: ClientInit 2026-06-08 Mon 05:46:02:048.473 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-08 Mon 05:46:02:048.485 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-08 Mon 05:46:02:048.508 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Received Ctrl Message with size 0 2026-06-08 Mon 05:46:02:048.676 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780897561035000 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780897561035900 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 87 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] max_sm_queue_count = 5 proxy seq 10 2026-06-08 Mon 05:46:02:048.716 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:46:02:048.747 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 11 2026-06-08 Mon 05:46:02:048.762 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:46:02:048.771 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:46:02:048.780 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:46:02:048.823 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Established 2026-06-08 Mon 05:46:09:108.236 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: OnMessage control in state: Established 2026-06-08 Mon 05:46:09:108.332 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Processing scm::EvSandeshMessageRecv in state Established 2026-06-08 Mon 05:46:09:108.345 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Established : EvSandeshMessageRecv 2026-06-08 Mon 05:46:09:108.391 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Received Ctrl Message with size 3 2026-06-08 Mon 05:46:09:108.408 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 with seqno 11, total = 0 2026-06-08 Mon 05:46:09:108.420 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 with seqno 13, total = 0 2026-06-08 Mon 05:46:09:108.434 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 with seqno 1, total = 0 2026-06-08 Mon 05:46:09:108.442 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:46:31:036.632 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463894287936, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 1 without seqno , total = 0 2026-06-08 Mon 05:47:31:037.349 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 2 without seqno , total = 0 2026-06-08 Mon 05:48:31:037.957 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 3 without seqno , total = 0 2026-06-08 Mon 05:49:22:816.012 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:43320 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:49:31:038.603 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 4 without seqno , total = 0 2026-06-08 Mon 05:49:35:951.697 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:39500 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:50:31:039.875 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 5 without seqno , total = 0 2026-06-08 Mon 05:51:25:828.472 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2398040 2026-06-08 Mon 05:51:25:828.534 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:51:25:828.587 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2398b00 2026-06-08 Mon 05:51:25:828.597 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:51:25:828.619 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2397560 2026-06-08 Mon 05:51:25:828.629 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:51:25:828.656 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2396a80 2026-06-08 Mon 05:51:25:828.665 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:51:25:828.687 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-08 Mon 05:51:25:828.693 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2395fa0 2026-06-08 Mon 05:51:25:828.699 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:51:25:829.072 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:51:25:829.120 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:51:25:829.134 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:51:25:829.146 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:51:25:829.168 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:51:25:867.779 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x23954c0 2026-06-08 Mon 05:51:25:867.841 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:51:25:867.911 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x23949e0 2026-06-08 Mon 05:51:25:867.933 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:51:25:868.119 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2393340 2026-06-08 Mon 05:51:25:868.137 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:51:25:868.170 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-08 Mon 05:51:25:868.177 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2392820 2026-06-08 Mon 05:51:25:868.185 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:51:25:868.209 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2393f00 2026-06-08 Mon 05:51:25:868.333 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:51:25:869.893 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:51:25:870.763 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:51:25:870.827 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:51:25:870.846 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:51:25:870.864 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:51:30:829.165 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:51:30:829.326 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:51:30:829.565 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:51:30:829.611 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:51:30:829.649 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:51:30:829.702 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:51:30:829.833 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:51:30:829.876 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:51:30:829.966 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:51:30:829.982 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:51:30:870.800 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:51:30:871.024 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:51:30:871.074 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:51:30:871.110 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:51:30:871.143 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:51:30:871.335 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:51:30:871.376 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:51:30:871.393 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:51:30:871.408 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:51:30:871.421 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:51:31:042.812 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 6 without seqno , total = 0 2026-06-08 Mon 05:51:35:829.762 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:51:35:829.884 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:51:35:829.933 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:51:35:829.991 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:51:35:830.025 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:51:35:830.223 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:51:35:830.249 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:51:35:830.273 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:51:35:830.292 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:51:35:830.308 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:51:35:871.420 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:51:35:871.565 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:51:35:871.600 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:51:35:871.639 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:51:35:871.664 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:51:35:871.873 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:51:35:871.901 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:51:35:871.913 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:51:35:871.936 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:51:35:871.948 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:51:35:910.488 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:43004::10.0.0.33:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 506 2026-06-08 Mon 05:51:35:910.525 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:43004::10.0.0.33:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:51:35:910.547 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:43004::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-08 Mon 05:51:35:910.555 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:43004::10.0.0.33:8086(-1):EvTcpClose 2026-06-08 Mon 05:51:35:911.664 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvTcpClose in state Established 2026-06-08 Mon 05:51:35:911.681 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Established : EvTcpClose 2026-06-08 Mon 05:51:35:917.170 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:51:35:917.312 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:51:35:917.424 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780897895911759 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Established successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = cn-jenkins-deploy-platform-ansible-os-5902-3. collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311703 calls = 35 average_bytes = 8905 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 106, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 36, ] ] ] 2026-06-08 Mon 05:51:35:917.447 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:51:35:917.455 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:51:35:917.472 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780897895911795 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:51:35:917.478 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:51:35:917.487 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvTcpDeleteSession in state Connect 2026-06-08 Mon 05:51:35:917.496 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:51:35:917.564 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780897895917288 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311703 calls = 35 average_bytes = 8905 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 106, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 36, ] ] ] 2026-06-08 Mon 05:51:35:917.577 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:51:35:922.134 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:52372::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:51:35:922.161 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:52372::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:51:35:922.167 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:52372::10.0.0.38:8086(-1):EvTcpConnected 2026-06-08 Mon 05:51:35:927.622 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:51:35:927.655 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:51:35:927.701 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: ClientInit 2026-06-08 Mon 05:51:35:927.717 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-1.:contrail-query-engine:0:Database count 2 2026-06-08 Mon 05:51:35:927.792 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:51:35:927.799 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:51:35:927.818 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SANDESH: Send FAILED: 1780897895927692 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:51:35:927.842 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:51:35:927.849 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:51:35:927.854 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:51:35:927.877 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:51:35:927.882 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:51:35:928.056 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SANDESH: Send FAILED: 1780897895927781 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 2 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311703 calls = 35 average_bytes = 8905 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 107, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 23, _iter59->first = sent _iter59->second = 36, ] ] ] 2026-06-08 Mon 05:51:35:928.073 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:51:35:948.950 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: OnMessage control in state: ClientInit 2026-06-08 Mon 05:51:35:949.001 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-08 Mon 05:51:35:949.009 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-08 Mon 05:51:35:949.039 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Received Ctrl Message with size 0 2026-06-08 Mon 05:51:35:949.176 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 2 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311703 calls = 35 average_bytes = 8905 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 54 dequeues = 54 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 62 dequeues = 62 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Established state_since = 1780897562048822 last_state = Established last_event = scm::EvSandeshSend last_event_at = 1780897885869099 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 34 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 289081 calls = 33 average_bytes = 8760 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 107, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 23, _iter59->first = sent _iter59->second = 36, ] ] max_sm_queue_count = 5 proxy seq 36 2026-06-08 Mon 05:51:35:949.203 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:51:35:949.221 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 17 2026-06-08 Mon 05:51:35:949.231 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:51:35:949.256 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.4 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] proxy seq 6 2026-06-08 Mon 05:51:35:949.264 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:51:35:949.270 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:51:35:949.302 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Established 2026-06-08 Mon 05:51:40:830.280 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:51:40:830.425 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:51:40:830.460 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:51:40:830.488 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:51:40:830.514 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:51:40:830.717 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:51:40:830.745 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:51:40:830.757 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:51:40:830.777 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:51:40:830.841 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:51:40:871.945 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:51:40:872.090 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:51:40:872.133 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:51:40:872.157 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:51:40:872.181 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:51:40:872.383 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x23954c0 2026-06-08 Mon 05:51:40:872.392 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:51:40:872.412 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:51:40:872.433 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x23949e0 2026-06-08 Mon 05:51:40:872.437 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:51:40:872.443 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:51:40:872.450 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2393340 2026-06-08 Mon 05:51:40:872.454 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:51:40:872.459 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:51:40:872.466 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2392820 2026-06-08 Mon 05:51:40:872.470 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:51:40:872.475 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:51:40:872.482 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2393f00 2026-06-08 Mon 05:51:40:872.486 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:51:40:872.491 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:51:40:872.684 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:51:40:872.697 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:51:40:872.737 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:51:40:872.754 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:51:40:872.764 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:51:40:872.777 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:51:40:872.787 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:51:40:872.809 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:51:40:872.816 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:51:40:872.826 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:51:40:872.832 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:51:40:877.387 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:51:40:877.428 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:51:40:877.441 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:51:40:877.452 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:51:45:830.800 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:51:45:830.916 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:51:45:830.961 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:51:45:830.988 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:51:45:831.011 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:51:45:831.287 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2398040 2026-06-08 Mon 05:51:45:831.322 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:51:45:831.350 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:51:45:831.366 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2398b00 2026-06-08 Mon 05:51:45:831.370 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:51:45:831.377 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:51:45:831.384 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2397560 2026-06-08 Mon 05:51:45:831.388 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:51:45:831.394 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:51:45:831.400 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2396a80 2026-06-08 Mon 05:51:45:831.404 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:51:45:831.410 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:51:45:831.416 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2395fa0 2026-06-08 Mon 05:51:45:831.420 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:51:45:831.426 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:51:45:831.750 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:51:45:831.767 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:51:45:831.780 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:51:45:831.789 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:51:45:831.803 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:51:45:831.812 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:51:45:831.839 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:51:45:831.847 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:51:45:831.857 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:51:45:831.863 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:51:45:831.881 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:51:45:834.672 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:51:45:834.704 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:51:45:834.713 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:51:45:834.722 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:52:31:049.309 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 7 without seqno , total = 0 2026-06-08 Mon 05:53:31:050.973 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 8 without seqno , total = 0 2026-06-08 Mon 05:53:39:365.568 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-08 Mon 05:53:39:365.623 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2392820 2026-06-08 Mon 05:53:39:365.635 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:365.681 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2393f00 2026-06-08 Mon 05:53:39:365.690 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:365.713 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x23949e0 2026-06-08 Mon 05:53:39:365.720 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:365.866 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2393340 2026-06-08 Mon 05:53:39:366.139 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:366.184 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x23954c0 2026-06-08 Mon 05:53:39:366.193 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:366.356 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:53:39:366.383 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:53:39:366.395 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:53:39:366.989 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:53:39:367.027 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:53:39:421.239 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2391d20 2026-06-08 Mon 05:53:39:421.430 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:421.485 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2391200 2026-06-08 Mon 05:53:39:421.499 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:421.526 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x23908e0 2026-06-08 Mon 05:53:39:421.537 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:421.565 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2390180 2026-06-08 Mon 05:53:39:421.576 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:421.608 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-08 Mon 05:53:39:421.616 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x238b1c0 2026-06-08 Mon 05:53:39:421.623 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:425.206 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-08 Mon 05:53:39:425.234 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2395fa0 2026-06-08 Mon 05:53:39:425.249 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:425.278 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2396a80 2026-06-08 Mon 05:53:39:425.285 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:425.306 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2397560 2026-06-08 Mon 05:53:39:425.312 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:425.333 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2398040 2026-06-08 Mon 05:53:39:425.350 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:425.382 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Disconnect status: -1 0x2398b00 2026-06-08 Mon 05:53:39:425.394 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-08 Mon 05:53:39:429.110 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:53:39:429.168 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:53:39:429.189 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:53:39:429.210 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:53:39:429.223 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:53:39:429.254 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:53:39:429.266 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:53:39:429.276 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:53:39:429.325 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:53:39:429.340 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:53:39:567.237 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:52372::10.0.0.38:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 506 2026-06-08 Mon 05:53:39:567.286 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:52372::10.0.0.38:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:53:39:567.311 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:52372::10.0.0.38:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-08 Mon 05:53:39:567.329 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:52372::10.0.0.38:8086(-1):EvTcpClose 2026-06-08 Mon 05:53:39:584.898 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvTcpClose in state Established 2026-06-08 Mon 05:53:39:584.930 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Established : EvTcpClose 2026-06-08 Mon 05:53:39:585.236 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-08 Mon 05:53:39:585.319 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:39:585.425 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898019585036 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Established successful_connections = 2 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = cn-jenkins-deploy-platform-ansible-os-5902-1. collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 473702 calls = 54 average_bytes = 8772 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 58, ] ] ] 2026-06-08 Mon 05:53:39:585.621 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:39:585.642 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:39:585.664 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898019585073 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:39:585.673 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:39:585.682 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvTcpDeleteSession in state Connect 2026-06-08 Mon 05:53:39:585.690 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:39:585.768 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898019585304 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 2 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 473702 calls = 54 average_bytes = 8772 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 58, ] ] ] 2026-06-08 Mon 05:53:39:585.778 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:39:589.275 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:52264::10.0.0.50:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:53:39:589.311 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:52264::10.0.0.50:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:53:39:589.322 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:52264::10.0.0.50:8086(-1):EvTcpConnected 2026-06-08 Mon 05:53:39:590.357 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:53:39:590.432 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:53:39:590.500 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: ClientInit 2026-06-08 Mon 05:53:39:590.521 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-1.:contrail-query-engine:0:Database count 3 2026-06-08 Mon 05:53:39:590.621 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:39:590.632 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:53:39:590.655 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SANDESH: Send FAILED: 1780898019590490 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:39:590.664 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:53:39:590.673 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:39:590.681 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:53:39:590.693 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:39:590.701 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:53:39:590.838 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SANDESH: Send FAILED: 1780898019590607 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 3 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 473702 calls = 54 average_bytes = 8772 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 28, _iter59->first = sent _iter59->second = 58, ] ] ] 2026-06-08 Mon 05:53:39:590.864 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:53:39:593.773 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:52264::10.0.0.50:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-08 Mon 05:53:39:593.796 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:52264::10.0.0.50:8086(-1):EvTcpClose 2026-06-08 Mon 05:53:39:594.740 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-06-08 Mon 05:53:39:594.764 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: ClientInit : EvTcpClose 2026-06-08 Mon 05:53:39:594.906 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Return to idle with 1000 2026-06-08 Mon 05:53:39:594.918 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Idle 2026-06-08 Mon 05:53:39:594.989 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:39:595.144 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898019594843 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 3 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 475275 calls = 55 average_bytes = 8641 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 59, ] ] ] 2026-06-08 Mon 05:53:39:595.172 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:39:595.185 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:39:595.207 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898019594881 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:39:595.216 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:39:595.226 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:53:39:595.236 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:39:595.359 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898019594962 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 3 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 475275 calls = 55 average_bytes = 8641 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 59, ] ] ] 2026-06-08 Mon 05:53:39:595.380 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:40:595.029 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:53:40:595.251 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-08 Mon 05:53:40:595.348 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:40:595.370 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: SANDESH: Send FAILED: 1780898020595094 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:40:595.378 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:40:595.385 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:40:595.482 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: SANDESH: Send FAILED: 1780898020595331 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 3 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 475275 calls = 55 average_bytes = 8641 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 33, _iter59->first = sent _iter59->second = 59, ] ] ] 2026-06-08 Mon 05:53:40:595.542 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:40:598.050 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:40818::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:53:40:598.074 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:40818::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:53:40:598.081 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:40818::10.0.0.33:8086(-1):EvTcpConnected 2026-06-08 Mon 05:53:40:598.804 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:53:40:598.820 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:53:40:598.866 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: ClientInit 2026-06-08 Mon 05:53:40:598.880 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-1.:contrail-query-engine:0:Database count 4 2026-06-08 Mon 05:53:40:598.948 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:40:598.954 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:53:40:598.978 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780898020598858 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:40:598.992 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:53:40:599.000 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:40:599.005 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:53:40:599.029 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:40:599.034 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:53:40:599.138 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780898020598938 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 4 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 475275 calls = 55 average_bytes = 8641 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 35, _iter59->first = sent _iter59->second = 59, ] ] ] 2026-06-08 Mon 05:53:40:599.155 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:53:42:509.034 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:40818::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-08 Mon 05:53:42:509.078 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:40818::10.0.0.33:8086(-1):EvTcpClose 2026-06-08 Mon 05:53:42:510.015 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-06-08 Mon 05:53:42:510.033 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: ClientInit : EvTcpClose 2026-06-08 Mon 05:53:42:510.156 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Return to idle with 1000 2026-06-08 Mon 05:53:42:510.165 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Idle 2026-06-08 Mon 05:53:42:510.201 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:42:510.316 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780898022510106 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 4 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 476848 calls = 56 average_bytes = 8515 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 37, _iter59->first = sent _iter59->second = 60, ] ] ] 2026-06-08 Mon 05:53:42:510.340 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:42:510.349 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:42:510.364 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780898022510138 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:42:510.378 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:42:510.386 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:53:42:510.395 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:42:510.463 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780898022510192 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 4 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 476848 calls = 56 average_bytes = 8515 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 37, _iter59->first = sent _iter59->second = 60, ] ] ] 2026-06-08 Mon 05:53:42:510.473 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:43:510.301 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:53:43:510.633 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:53:43:510.756 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:43:510.793 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SANDESH: Send FAILED: 1780898023510402 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:43:510.802 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:43:510.814 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:43:510.998 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SANDESH: Send FAILED: 1780898023510736 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 4 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 476848 calls = 56 average_bytes = 8515 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 40, _iter59->first = sent _iter59->second = 60, ] ] ] 2026-06-08 Mon 05:53:43:511.025 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:43:512.229 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:35700::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:53:43:512.260 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:35700::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:53:43:512.269 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:35700::10.0.0.38:8086(-1):EvTcpConnected 2026-06-08 Mon 05:53:43:512.341 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:53:43:512.366 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:53:43:512.400 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: ClientInit 2026-06-08 Mon 05:53:43:512.418 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-1.:contrail-query-engine:0:Database count 5 2026-06-08 Mon 05:53:43:512.491 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:43:512.498 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:53:43:512.514 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SANDESH: Send FAILED: 1780898023512393 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:43:512.520 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:53:43:512.526 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:43:512.531 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:53:43:512.540 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:43:512.545 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:53:43:512.625 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SANDESH: Send FAILED: 1780898023512479 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 5 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 476848 calls = 56 average_bytes = 8515 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 170, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 42, _iter59->first = sent _iter59->second = 60, ] ] ] 2026-06-08 Mon 05:53:43:512.647 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:53:43:513.361 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:35700::10.0.0.38:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-08 Mon 05:53:43:513.385 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:35700::10.0.0.38:8086(-1):EvTcpClose 2026-06-08 Mon 05:53:43:513.574 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-06-08 Mon 05:53:43:513.592 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: ClientInit : EvTcpClose 2026-06-08 Mon 05:53:43:513.678 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Return to idle with 1000 2026-06-08 Mon 05:53:43:513.685 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Idle 2026-06-08 Mon 05:53:43:513.723 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:43:513.822 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SANDESH: Send FAILED: 1780898023513642 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 5 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 478421 calls = 57 average_bytes = 8393 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 170, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 44, _iter59->first = sent _iter59->second = 61, ] ] ] 2026-06-08 Mon 05:53:43:513.844 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:43:513.852 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:43:513.866 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SANDESH: Send FAILED: 1780898023513665 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:43:513.872 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:43:513.878 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:53:43:513.902 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:43:513.993 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SANDESH: Send FAILED: 1780898023513714 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 478421 calls = 57 average_bytes = 8393 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 170, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 44, _iter59->first = sent _iter59->second = 61, ] ] ] 2026-06-08 Mon 05:53:43:514.010 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:44:366.451 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:53:44:366.658 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:53:44:366.707 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:53:44:366.962 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:53:44:367.059 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:53:44:367.087 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:53:44:367.099 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:53:44:367.177 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:53:44:367.394 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:53:44:367.411 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:53:44:429.235 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:53:44:429.389 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:53:44:429.442 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:53:44:429.499 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:53:44:429.527 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:53:44:429.538 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:53:44:429.547 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:53:44:429.583 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:53:44:429.605 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:53:44:429.632 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:53:44:429.662 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:53:44:429.688 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:53:44:429.714 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:53:44:429.743 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:53:44:429.758 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:53:44:429.866 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:53:44:429.877 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:53:44:429.889 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:53:44:429.899 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:53:44:429.909 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:53:44:513.843 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:53:44:514.110 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-08 Mon 05:53:44:514.223 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:44:514.248 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: SANDESH: Send FAILED: 1780898024513933 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:44:514.263 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:44:514.271 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:44:514.370 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: SANDESH: Send FAILED: 1780898024514206 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 478421 calls = 57 average_bytes = 8393 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 170, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 47, _iter59->first = sent _iter59->second = 61, ] ] ] 2026-06-08 Mon 05:53:44:514.388 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:44:515.896 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:43026::10.0.0.50:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:53:44:515.920 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:43026::10.0.0.50:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:53:44:515.927 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:43026::10.0.0.50:8086(-1):EvTcpConnected 2026-06-08 Mon 05:53:44:516.003 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:53:44:516.032 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:53:44:516.069 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: ClientInit 2026-06-08 Mon 05:53:44:516.085 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-1.:contrail-query-engine:0:Database count 6 2026-06-08 Mon 05:53:44:516.141 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:44:516.148 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:53:44:516.163 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: SANDESH: Send FAILED: 1780898024516063 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:44:516.169 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:53:44:516.175 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:44:516.180 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:53:44:516.188 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:44:516.192 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:53:44:516.279 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: SANDESH: Send FAILED: 1780898024516133 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 478421 calls = 57 average_bytes = 8393 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 171, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 49, _iter59->first = sent _iter59->second = 61, ] ] ] 2026-06-08 Mon 05:53:44:516.298 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:53:49:367.110 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:53:49:367.270 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:53:49:367.319 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:53:49:367.420 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:53:49:367.455 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:53:49:367.512 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:53:49:367.540 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:53:49:367.555 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:53:49:367.660 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:53:49:371.597 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:53:49:429.579 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:53:49:429.740 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:53:49:429.785 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:53:49:429.838 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:53:49:429.866 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:53:49:429.883 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:53:49:429.892 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:53:49:429.919 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:53:49:429.962 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:53:49:429.983 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:53:49:429.992 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:53:49:430.024 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:53:49:430.045 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:53:49:430.064 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:53:49:430.087 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:53:49:430.289 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:53:49:430.316 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:53:49:430.328 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:53:49:430.340 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:53:49:430.352 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:53:51:301.018 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:43026::10.0.0.50:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 506 2026-06-08 Mon 05:53:51:301.073 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:43026::10.0.0.50:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:53:51:301.103 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:43026::10.0.0.50:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-08 Mon 05:53:51:301.114 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:43026::10.0.0.50:8086(-1):EvTcpClose 2026-06-08 Mon 05:53:51:301.181 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-06-08 Mon 05:53:51:301.198 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: ClientInit : EvTcpClose 2026-06-08 Mon 05:53:51:301.376 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Return to idle with 1000 2026-06-08 Mon 05:53:51:301.397 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Idle 2026-06-08 Mon 05:53:51:301.471 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:51:301.625 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SANDESH: Send FAILED: 1780898031301295 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 51, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:53:51:303.696 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:51:303.723 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:51:303.752 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SANDESH: Send FAILED: 1780898031301339 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:51:303.761 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:51:303.772 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:53:51:303.785 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:51:303.926 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SANDESH: Send FAILED: 1780898031301451 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 51, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:53:51:303.954 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:52:301.555 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:53:52:301.984 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-08 Mon 05:53:52:302.134 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:52:302.163 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:53:52:302.174 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SANDESH: Send FAILED: 1780898032301788 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:52:302.222 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:52:302.249 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:52:302.415 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SANDESH: Send FAILED: 1780898032302108 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 54, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:53:52:302.435 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:52:302.454 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:53:52:303.342 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:53:52:303.364 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:53:52:303.530 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:53:52:303.578 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Idle 2026-06-08 Mon 05:53:52:303.649 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:52:303.829 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898032303440 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 56, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:53:52:303.856 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:52:303.867 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:52:303.884 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898032303475 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:53:52:303.890 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:52:303.898 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:53:52:303.934 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:52:304.051 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898032303634 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 174, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 56, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:53:52:304.070 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:54:367.577 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:53:54:367.714 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:53:54:367.771 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:53:54:367.805 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:53:54:367.930 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:53:54:367.958 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:53:54:367.999 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:53:54:368.013 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:53:54:371.671 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:53:54:371.948 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:53:54:429.900 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:53:54:430.060 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:53:54:430.084 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:53:54:430.114 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:53:54:430.140 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:53:54:430.167 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:53:54:430.196 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:53:54:430.208 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:53:54:430.219 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:53:54:430.228 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:53:54:430.317 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:53:54:430.361 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:53:54:430.387 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:53:54:430.414 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:53:54:430.443 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:53:54:430.573 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:53:54:430.606 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:53:54:430.618 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:53:54:430.628 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:53:54:430.637 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:53:59:368.097 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:53:59:368.306 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:53:59:368.361 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:53:59:368.410 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:53:59:368.528 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:53:59:368.562 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:53:59:368.574 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:53:59:368.606 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:53:59:372.757 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:53:59:373.269 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:53:59:430.156 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:53:59:430.417 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:53:59:430.451 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:53:59:430.484 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:53:59:430.513 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:53:59:430.542 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:53:59:430.574 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:53:59:430.593 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:53:59:430.604 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:53:59:430.614 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:53:59:430.627 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:53:59:430.665 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:53:59:430.689 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:53:59:430.717 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:53:59:430.743 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:53:59:430.984 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:53:59:431.018 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:53:59:431.035 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:53:59:431.053 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:53:59:431.087 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:01:051.341 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:01:051.471 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780898041051189 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 121 dequeues = 121 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 153 dequeues = 153 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780898032303577 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780898032303933 ] max_sm_queue_count = 6 ] 2026-06-08 Mon 05:54:01:051.488 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:01:051.501 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:01:051.636 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780898041051290 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 174, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 59, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:54:01:051.671 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:04:368.600 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:04:368.759 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:04:368.793 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:04:368.818 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:04:369.160 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:54:04:369.210 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:54:04:369.227 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:54:04:369.241 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:54:04:373.337 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:04:373.748 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:54:04:430.486 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:04:430.674 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:54:04:430.703 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:04:430.736 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:04:430.765 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:04:430.790 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:04:430.826 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:54:04:430.840 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:54:04:430.851 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:54:04:430.861 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:54:04:431.018 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:04:431.072 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:04:431.103 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:04:431.130 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:04:431.153 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:04:431.305 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:54:04:431.337 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:54:04:431.350 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:54:04:431.361 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:54:04:431.373 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:06:044.191 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:06:044.303 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SANDESH: Send FAILED: 1780898046044135 [SYS_NOTICE]: QEDbStatsUve: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.4 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] ] 2026-06-08 Mon 05:54:06:044.316 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: QEDbStatsUve 2026-06-08 Mon 05:54:08:303.731 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:54:08:304.100 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:54:08:304.192 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:54:08:304.214 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: SANDESH: Send FAILED: 1780898048303815 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:54:08:304.222 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:54:08:304.239 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:54:08:304.382 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: SANDESH: Send FAILED: 1780898048304178 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 174, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 62, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:54:08:304.399 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:08:304.427 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:54:08:304.434 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:54:08:304.494 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:54:08:304.532 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:54:08:304.655 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:54:08:304.690 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Idle 2026-06-08 Mon 05:54:08:304.748 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:08:304.892 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898048304603 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 174, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 64, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:54:08:304.917 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:08:304.926 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:08:304.942 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898048304636 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:54:08:304.949 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:54:08:304.958 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:54:08:305.021 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:08:305.118 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898048304737 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 175, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 64, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:54:08:305.132 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:09:369.278 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:09:369.509 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:09:369.554 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:09:369.603 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:09:369.936 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:54:09:370.006 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:54:09:370.025 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:54:09:370.044 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:54:09:373.858 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:09:374.416 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:54:09:430.882 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:09:431.356 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:09:431.591 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:54:09:431.644 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:54:09:431.661 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:09:431.842 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:09:431.904 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:09:431.944 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:09:432.117 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:09:432.311 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:09:432.354 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:09:432.388 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:09:432.539 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:54:09:432.564 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:54:09:432.596 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:54:09:432.611 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:54:09:432.625 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:54:09:432.638 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:54:09:432.656 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:54:09:432.701 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:14:370.059 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:14:370.249 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:14:370.298 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:14:370.339 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:14:370.674 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:54:14:370.741 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:54:14:370.762 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:54:14:370.778 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:54:14:374.503 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:14:374.956 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:54:14:431.686 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:14:431.855 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:14:431.914 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:54:14:431.942 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:54:14:432.590 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:14:432.700 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:14:432.768 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:54:14:432.793 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:54:14:432.808 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:14:432.848 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:14:432.893 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:14:432.929 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:14:432.959 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:14:432.997 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:14:433.040 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:54:14:433.205 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:54:14:433.229 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:54:14:433.247 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:54:14:433.265 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:14:433.297 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:54:19:370.788 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:19:370.970 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:19:371.020 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:19:371.052 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:19:371.251 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:54:19:371.281 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:54:19:371.296 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:54:19:371.312 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:54:19:375.035 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:19:375.437 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:54:19:431.996 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:19:432.153 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:19:432.211 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:54:19:432.237 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:54:19:432.815 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:19:432.896 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:54:19:432.913 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:19:432.950 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:54:19:433.065 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:19:433.110 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:54:19:433.233 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:19:433.293 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:19:433.325 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:19:433.362 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:19:433.401 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:19:433.526 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:54:19:433.572 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:54:19:433.595 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:54:19:433.615 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:19:433.626 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:54:24:371.332 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:24:371.476 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:24:371.519 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:24:371.548 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:24:371.785 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:54:24:371.807 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:54:24:371.820 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:54:24:371.830 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:54:24:375.497 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:24:375.846 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:54:24:432.274 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:24:432.405 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:24:432.449 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:54:24:432.470 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:54:24:432.932 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:24:433.021 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:54:24:433.038 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:24:433.069 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:54:24:433.130 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:24:433.161 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:54:24:433.566 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:24:433.625 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:24:433.650 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:24:433.676 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:24:433.701 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:24:433.911 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:54:24:433.959 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:54:24:434.045 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:54:24:434.069 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:54:24:434.081 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:29:371.861 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:29:372.113 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:29:372.164 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:29:372.206 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:29:372.466 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:54:29:372.513 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:54:29:372.528 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:54:29:372.541 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:54:29:375.917 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:29:376.410 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:54:29:434.056 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:29:434.187 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:29:434.218 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:29:434.244 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:29:434.270 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:29:434.294 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:29:434.340 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:29:434.381 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:54:29:434.400 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:54:29:434.411 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:54:29:434.421 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:54:29:434.431 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:54:29:434.439 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:29:434.462 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:29:434.483 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:29:434.576 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:54:29:434.594 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:54:29:434.647 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:54:29:434.661 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:54:29:434.672 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:31:052.426 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 9 without seqno , total = 0 2026-06-08 Mon 05:54:31:052.504 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:31:052.568 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898071052276 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 129 dequeues = 129 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 164 dequeues = 164 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780898048304689 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780898048305019 ] max_sm_queue_count = 6 ] 2026-06-08 Mon 05:54:31:052.598 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:31:052.609 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:31:052.746 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: SANDESH: Send FAILED: 1780898071052401 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 175, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 67, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:54:31:052.766 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463794914880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:34:372.567 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:34:372.726 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:34:372.762 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:34:372.797 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:34:373.132 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:54:34:373.175 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:54:34:373.195 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:54:34:373.225 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:54:34:376.465 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:34:376.826 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:54:34:434.452 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:34:434.612 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:34:434.646 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:34:434.690 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:54:34:434.713 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:54:34:434.724 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:54:34:434.732 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:34:434.760 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:34:434.792 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:34:434.828 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:34:434.854 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:34:434.880 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:34:434.919 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:54:34:434.938 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:54:34:434.947 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:34:435.099 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:54:34:435.116 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:54:34:435.135 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:54:34:435.147 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:54:34:435.159 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:38:305.303 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:54:38:305.799 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-08 Mon 05:54:38:305.918 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:54:38:305.965 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SANDESH: Send FAILED: 1780898078305605 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:54:38:305.975 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:54:38:305.987 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:54:38:306.181 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SANDESH: Send FAILED: 1780898078305901 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 175, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 69, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:54:38:306.209 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:38:306.580 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:54:38:306.610 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:54:38:306.855 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:54:38:306.887 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:54:38:307.042 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:54:38:307.082 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Idle 2026-06-08 Mon 05:54:38:307.137 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:38:307.276 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: SANDESH: Send FAILED: 1780898078306963 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 175, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 71, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:54:38:307.295 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:38:307.306 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:38:307.327 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: SANDESH: Send FAILED: 1780898078307019 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:54:38:307.339 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:54:38:307.348 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:54:38:307.388 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:38:307.505 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: SANDESH: Send FAILED: 1780898078307125 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 176, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 71, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:54:38:307.522 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463786518080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:39:373.204 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:39:373.320 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:39:373.362 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:39:373.396 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:39:373.563 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:54:39:373.589 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:54:39:373.601 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:54:39:373.635 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:54:39:376.874 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:39:377.187 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:54:39:434.759 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:39:434.894 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:39:434.944 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:54:39:434.968 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:54:39:434.977 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:39:435.019 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:54:39:435.031 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:39:435.055 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:39:435.084 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:54:39:435.100 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:54:39:435.127 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:39:435.168 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:39:435.192 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:39:435.222 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:39:435.254 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:39:435.527 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:54:39:435.561 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:54:39:435.574 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:54:39:435.587 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:54:39:435.600 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:44:378.175 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:44:378.310 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:44:378.355 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:44:378.390 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:44:378.422 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:44:378.894 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:54:44:378.935 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:54:44:378.948 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:54:44:378.961 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:54:44:378.974 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:54:44:435.033 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:44:435.165 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:44:435.211 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:54:44:435.232 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:54:44:435.241 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:44:435.266 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:44:435.291 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:44:435.319 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:54:44:435.331 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:54:44:435.341 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:54:44:435.572 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:44:435.637 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:44:435.660 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:44:435.682 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:44:435.712 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:44:435.907 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:54:44:435.930 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:54:44:435.941 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:54:44:435.953 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:54:44:435.964 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:49:378.992 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:49:379.148 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:49:379.191 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:49:379.223 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:49:379.253 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:49:379.444 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:54:49:379.472 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:54:49:379.485 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:54:49:379.508 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:54:49:386.571 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:54:49:435.272 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:49:435.423 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:49:435.474 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:54:49:435.496 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:54:49:435.505 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:49:435.530 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:49:435.553 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:49:435.591 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:54:49:435.603 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:54:49:435.613 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:54:49:435.945 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:49:436.022 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:49:436.053 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:49:436.072 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:49:436.092 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:49:436.426 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:54:49:436.454 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:54:49:436.467 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:54:49:436.478 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:54:49:436.489 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:54:380.059 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:54:380.246 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:54:380.292 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:54:380.329 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:54:380.478 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393f00 2026-06-08 Mon 05:54:54:380.521 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2392820 2026-06-08 Mon 05:54:54:380.539 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23949e0 2026-06-08 Mon 05:54:54:380.556 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23954c0 2026-06-08 Mon 05:54:54:386.661 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:54:387.055 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2393340 2026-06-08 Mon 05:54:54:435.546 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:54:435.717 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:54:435.764 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391d20 2026-06-08 Mon 05:54:54:435.809 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2391200 2026-06-08 Mon 05:54:54:435.819 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:54:435.850 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:54:435.885 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:54:435.925 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x23908e0 2026-06-08 Mon 05:54:54:435.943 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2390180 2026-06-08 Mon 05:54:54:435.956 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x238b1c0 2026-06-08 Mon 05:54:54:436.471 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:54:436.544 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:54:436.570 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:54:436.597 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:54:436.623 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:54:436.849 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2396a80 2026-06-08 Mon 05:54:54:436.882 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2397560 2026-06-08 Mon 05:54:54:436.894 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398040 2026-06-08 Mon 05:54:54:436.907 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2395fa0 2026-06-08 Mon 05:54:54:436.917 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: -1 0x2398b00 2026-06-08 Mon 05:54:59:380.574 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393f00 2026-06-08 Mon 05:54:59:380.729 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2392820 2026-06-08 Mon 05:54:59:380.764 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23949e0 2026-06-08 Mon 05:54:59:380.797 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23954c0 2026-06-08 Mon 05:54:59:380.979 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2393f00 2026-06-08 Mon 05:54:59:381.013 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:381.048 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:381.074 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2392820 2026-06-08 Mon 05:54:59:381.092 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:381.099 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:381.106 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x23949e0 2026-06-08 Mon 05:54:59:381.110 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:381.115 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:381.122 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x23954c0 2026-06-08 Mon 05:54:59:381.127 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:381.133 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:381.409 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:381.430 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:381.452 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:381.463 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:381.469 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:381.478 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:381.484 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:381.493 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:381.500 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:381.918 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463769724480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:59:381.947 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463769724480, Pid 1]: SANDESH: Send FAILED: 1780898099381528 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:54:59:381.961 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463769724480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:54:59:382.042 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:382.060 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:382.071 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:387.126 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2393340 2026-06-08 Mon 05:54:59:387.534 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2393340 2026-06-08 Mon 05:54:59:387.559 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:387.585 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:387.779 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:387.805 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:387.832 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:435.847 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391d20 2026-06-08 Mon 05:54:59:436.105 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2391200 2026-06-08 Mon 05:54:59:436.169 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2391d20 2026-06-08 Mon 05:54:59:436.176 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:436.196 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:436.206 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2391200 2026-06-08 Mon 05:54:59:436.210 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:436.216 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:436.234 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x23908e0 2026-06-08 Mon 05:54:59:436.276 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2390180 2026-06-08 Mon 05:54:59:436.309 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x238b1c0 2026-06-08 Mon 05:54:59:436.390 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x23908e0 2026-06-08 Mon 05:54:59:436.396 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:436.404 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:436.410 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2390180 2026-06-08 Mon 05:54:59:436.414 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:436.420 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:436.426 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x238b1c0 2026-06-08 Mon 05:54:59:436.430 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:436.435 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:436.564 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:436.576 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:436.588 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:436.595 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:436.679 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:436.935 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463769724480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:59:436.970 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463769724480, Pid 1]: SANDESH: Send FAILED: 1780898099436710 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:54:59:436.986 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463769724480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:54:59:437.018 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:437.040 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:437.048 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:437.067 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:437.078 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:437.084 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:437.094 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:437.100 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:437.106 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2396a80 2026-06-08 Mon 05:54:59:437.169 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2397560 2026-06-08 Mon 05:54:59:437.197 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398040 2026-06-08 Mon 05:54:59:437.218 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2395fa0 2026-06-08 Mon 05:54:59:437.244 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2398b00 2026-06-08 Mon 05:54:59:437.279 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:437.289 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:437.417 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2396a80 2026-06-08 Mon 05:54:59:437.423 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:437.435 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:437.447 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2397560 2026-06-08 Mon 05:54:59:437.451 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:437.456 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:437.479 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2398040 2026-06-08 Mon 05:54:59:437.483 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:437.489 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:437.496 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2398b00 2026-06-08 Mon 05:54:59:437.499 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:437.505 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:437.532 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: RAC_Connect status: 0 0x2395fa0 2026-06-08 Mon 05:54:59:437.537 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:54:59:437.543 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-08 Mon 05:54:59:437.861 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:437.893 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:437.911 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:437.927 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:437.937 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:437.953 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:437.963 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:437.976 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:437.992 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:438.014 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-08 Mon 05:54:59:438.024 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-08 Mon 05:54:59:438.569 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:59:438.598 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: SANDESH: Send FAILED: 1780898099438056 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:54:59:438.604 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463890089536, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:54:59:438.621 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:438.633 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:438.641 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:54:59:438.649 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-08 Mon 05:55:01:054.335 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:55:01:054.433 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780898101054197 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 139 dequeues = 139 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 177 dequeues = 177 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780898078307081 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780898099438566 ] max_sm_queue_count = 6 ] 2026-06-08 Mon 05:55:01:054.461 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:55:01:054.472 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:55:01:054.613 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780898101054302 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 236, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 77, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:55:01:054.628 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:55:06:050.700 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:55:06:050.776 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SANDESH: Send FAILED: 1780898106050667 [SYS_NOTICE]: QEDbStatsUve: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.4 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] ] 2026-06-08 Mon 05:55:06:050.796 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: QEDbStatsUve 2026-06-08 Mon 05:55:08:307.461 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:55:08:307.778 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-08 Mon 05:55:08:307.879 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:55:08:307.904 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: SANDESH: Send FAILED: 1780898108307566 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:55:08:307.912 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:55:08:307.920 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:55:08:308.078 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: SANDESH: Send FAILED: 1780898108307866 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 236, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 80, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:55:08:308.118 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463803311680, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:55:08:308.328 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:55:08:308.361 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:55:08:308.604 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:55:08:308.627 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:55:08:308.765 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:55:08:308.812 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Idle 2026-06-08 Mon 05:55:08:308.870 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:55:08:308.959 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: SANDESH: Send FAILED: 1780898108308708 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 236, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 82, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:55:08:308.985 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:55:08:308.993 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:55:08:309.027 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: SANDESH: Send FAILED: 1780898108308740 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:55:08:309.033 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:55:08:309.040 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:55:08:309.077 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:55:08:309.160 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: SANDESH: Send FAILED: 1780898108308858 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 237, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 82, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:55:08:309.175 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:55:31:057.116 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 10 without seqno , total = 0 2026-06-08 Mon 05:55:31:057.190 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:55:31:057.263 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780898131056944 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 147 dequeues = 147 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 188 dequeues = 188 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780898108308811 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780898108309076 ] max_sm_queue_count = 6 ] 2026-06-08 Mon 05:55:31:057.279 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:55:31:057.291 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:55:31:057.670 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: SANDESH: Send FAILED: 1780898131057090 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 237, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 85, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:55:31:057.714 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463778121280, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:55:38:310.741 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:55:38:311.031 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:55:38:311.126 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:55:38:311.149 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SANDESH: Send FAILED: 1780898138310822 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:55:38:311.157 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:55:38:311.165 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:55:38:311.258 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SANDESH: Send FAILED: 1780898138311112 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 237, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 87, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:55:38:311.282 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:55:38:313.510 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:58970::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:55:38:313.538 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:58970::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:55:38:313.545 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463905628672, Pid 1]: 10.0.0.38:58970::10.0.0.38:8086(-1):EvTcpConnected 2026-06-08 Mon 05:55:38:313.590 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:55:38:313.614 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:55:38:313.660 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: ClientInit 2026-06-08 Mon 05:55:38:313.679 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-1.:contrail-query-engine:0:Database count 7 2026-06-08 Mon 05:55:38:313.772 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:55:38:313.781 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:55:38:313.804 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: SANDESH: Send FAILED: 1780898138313651 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:55:38:313.820 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:55:38:313.830 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:55:38:313.837 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:55:38:313.849 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:55:38:313.856 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:55:38:314.014 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: SANDESH: Send FAILED: 1780898138313757 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 7 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 238, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 89, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-08 Mon 05:55:38:314.031 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:55:38:319.914 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463765526080, Pid 1]: OnMessage control in state: ClientInit 2026-06-08 Mon 05:55:38:320.001 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-08 Mon 05:55:38:320.023 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-08 Mon 05:55:38:320.046 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Received Ctrl Message with size 0 2026-06-08 Mon 05:55:38:321.088 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 7 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479994 calls = 58 average_bytes = 8275 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 147 dequeues = 147 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 188 dequeues = 188 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780898108308811 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780898108309076 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 17 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 130513 calls = 14 average_bytes = 9322 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 9 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 238, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 89, _iter59->first = sent _iter59->second = 62, ] ] max_sm_queue_count = 6 proxy seq 84 2026-06-08 Mon 05:55:38:321.153 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:55:38:321.176 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 49 2026-06-08 Mon 05:55:38:321.187 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:55:38:321.227 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.4 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] proxy seq 10 2026-06-08 Mon 05:55:38:321.237 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:55:38:321.243 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:55:38:321.288 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463773922880, Pid 1]: Established 2026-06-08 Mon 05:56:31:057.726 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463790716480, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 11 without seqno , total = 0 2026-06-08 Mon 05:57:31:058.826 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463782319680, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 12 without seqno , total = 0 2026-06-08 Mon 05:57:46:702.713 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463818405440, Pid 1]: Processing scm::EvStop in state Established 2026-06-08 Mon 05:57:46:702.857 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463818405440, Pid 1]: Idle 2026-06-08 Mon 05:57:46:702.954 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463818405440, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:57:46:702.972 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463818405440, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:57:46:703.076 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463818405440, Pid 1]: SANDESH: Send FAILED: 1780898266702941 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1780897510667513 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, ] ] rx_socket_stats= [ bytes = 4481 calls = 8 average_bytes = 560 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 628058 calls = 72 average_bytes = 8723 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 16 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 238, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 91, _iter59->first = sent _iter59->second = 79, ] ] ] 2026-06-08 Mon 05:57:46:703.094 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140463818405440, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:58:11:312.545 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: http-server-port 8091 2026-06-08 Mon 05:58:11:312.624 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Max-tasks 16 2026-06-08 Mon 05:58:11:312.632 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Max-slice 100 2026-06-08 Mon 05:58:11:312.639 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Collectors 10.0.0.38:8086 2026-06-08 Mon 05:58:11:312.644 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Collectors 10.0.0.50:8086 2026-06-08 Mon 05:58:11:312.695 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Collectors 10.0.0.33:8086 2026-06-08 Mon 05:58:11:312.721 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: ROLE : Generator 2026-06-08 Mon 05:58:11:312.728 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: MODULE : contrail-query-engine 2026-06-08 Mon 05:58:11:312.748 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-5902-1. 2026-06-08 Mon 05:58:11:312.754 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: NODE TYPE : Database 2026-06-08 Mon 05:58:11:312.760 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: INSTANCE ID : 0 2026-06-08 Mon 05:58:11:312.766 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: HTTP SERVER PORT : 8091 2026-06-08 Mon 05:58:11:312.773 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-06-08 Mon 05:58:11:316.781 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: HTTP Introspect Init 2026-06-08 Mon 05:58:11:316.921 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8091 Initialization complete src/contrail-common/io/tcp_server.cc 138 2026-06-08 Mon 05:58:11:317.022 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Sandesh Http Server Port: 8091 2026-06-08 Mon 05:58:11:317.103 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: NOT Writing http_port 8091TO : /tmp/contrail-query-engine.0.http_port 2026-06-08 Mon 05:58:11:317.113 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: CONNECT TO COLLECTOR: 1 2026-06-08 Mon 05:58:11:317.529 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Idle 2026-06-08 Mon 05:58:11:317.704 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: No Client: 1780898291317628 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780898291317541 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:58:11:317.855 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: Logging: DISABLED -> ENABLED 2026-06-08 Mon 05:58:11:317.880 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-06-08 Mon 05:58:11:317.893 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Cassandra Servers: 10.0.0.38:9042 10.0.0.50:9042 10.0.0.33:9042 2026-06-08 Mon 05:58:11:318.010 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Processing scm::EvStart in state Idle 2026-06-08 Mon 05:58:11:318.101 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Disconnect 2026-06-08 Mon 05:58:11:318.157 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Processing scm::EvCollectorUpdate in state Disconnect 2026-06-08 Mon 05:58:11:318.346 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-08 Mon 05:58:11:318.412 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:58:11:318.435 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: SANDESH: Send FAILED: 1780898291318094 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-08 Mon 05:58:11:318.444 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:58:11:318.454 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:58:11:320.900 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: SANDESH: Send FAILED: 1780898291318147 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780898291317541 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:58:11:320.948 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:58:11:320.963 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:58:11:321.013 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: SANDESH: Send FAILED: 1780898291318184 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780898291317541 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:58:11:321.024 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:58:11:321.031 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:58:11:321.054 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: SANDESH: Send FAILED: 1780898291318199 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-08 Mon 05:58:11:321.060 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:58:11:321.066 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:58:11:321.102 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: SANDESH: Send FAILED: 1780898291318393 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780898291317541 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:58:11:321.112 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:58:11:350.140 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-06-08 Mon 05:58:11:354.337 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069388666432, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:58:11:354.396 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069388666432, Pid 1]: SANDESH: Send FAILED: 1780898291353788 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-08 Mon 05:58:11:354.404 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069388666432, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:58:11:356.417 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:48230::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-06-08 Mon 05:58:11:356.453 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: 10.0.0.38:48230::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:58:11:356.462 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: 10.0.0.38:48230::10.0.0.33:8086(-1):EvTcpConnected 2026-06-08 Mon 05:58:11:356.514 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdb7540 2026-06-08 Mon 05:58:11:356.522 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.538 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.555 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdbc540 2026-06-08 Mon 05:58:11:356.562 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.572 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.585 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdbcdf0 2026-06-08 Mon 05:58:11:356.591 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.601 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.625 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdbdab0 2026-06-08 Mon 05:58:11:356.632 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.641 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.653 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdbe770 2026-06-08 Mon 05:58:11:356.660 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.669 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.681 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdbf430 2026-06-08 Mon 05:58:11:356.687 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.696 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.707 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdc0c30 2026-06-08 Mon 05:58:11:356.713 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.723 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.735 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdbfff0 2026-06-08 Mon 05:58:11:356.741 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.750 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.761 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdc1870 2026-06-08 Mon 05:58:11:356.767 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.777 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.788 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdc24b0 2026-06-08 Mon 05:58:11:356.794 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.804 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.815 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdc30f0 2026-06-08 Mon 05:58:11:356.822 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.836 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.848 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdc55b0 2026-06-08 Mon 05:58:11:356.854 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.863 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.874 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdc61f0 2026-06-08 Mon 05:58:11:356.880 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.889 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.900 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdc4970 2026-06-08 Mon 05:58:11:356.907 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.915 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:356.926 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: RAC_Connect status: 0 0xdc3d30 2026-06-08 Mon 05:58:11:356.932 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: Connected to REDIS... 2026-06-08 Mon 05:58:11:356.941 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-08 Mon 05:58:11:357.180 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:58:11:358.659 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:58:11:358.778 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: ClientInit 2026-06-08 Mon 05:58:11:358.802 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-1.:contrail-query-engine:0:Database count 1 2026-06-08 Mon 05:58:11:358.907 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:58:11:358.918 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:58:11:358.942 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: SANDESH: Send FAILED: 1780898291358769 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-08 Mon 05:58:11:358.960 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:58:11:358.970 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:58:11:358.977 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:58:11:359.002 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:58:11:359.009 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:58:11:359.143 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: SANDESH: Send FAILED: 1780898291358889 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780898291317541 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 18, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:58:11:359.164 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:58:11:372.572 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:372.612 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:372.727 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:58:11:372.740 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:58:11:372.769 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: SANDESH: Send FAILED: 1780898291372667 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-08 Mon 05:58:11:372.779 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:58:11:374.516 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:374.565 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:374.586 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:374.604 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:374.614 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:374.631 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:374.641 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:374.658 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:374.668 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:374.746 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:58:11:374.758 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:58:11:374.784 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: SANDESH: Send FAILED: 1780898291374710 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:58:11:374.793 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:58:11:377.957 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:378.000 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:378.020 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:378.037 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:378.090 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:378.101 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:378.117 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:378.128 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:378.144 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:378.154 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:378.170 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:378.180 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:378.196 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:378.206 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:378.223 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:378.234 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:378.249 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:378.259 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:378.274 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:378.285 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:378.300 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:378.314 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:378.329 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-08 Mon 05:58:11:378.340 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-08 Mon 05:58:11:378.485 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: OnMessage control in state: ClientInit 2026-06-08 Mon 05:58:11:378.515 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-08 Mon 05:58:11:378.524 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-08 Mon 05:58:11:378.546 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Received Ctrl Message with size 0 2026-06-08 Mon 05:58:11:378.675 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780898291317541 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 18, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] proxy seq 5 2026-06-08 Mon 05:58:11:378.702 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:58:11:378.729 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 6 2026-06-08 Mon 05:58:11:378.743 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:58:11:378.751 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:58:11:378.760 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:58:11:378.793 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: Established 2026-06-08 Mon 05:58:11:381.375 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:386.458 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:386.507 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:386.524 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:386.540 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:386.573 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:386.619 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:386.634 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:386.648 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:58:11:386.663 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-08 Mon 05:59:11:318.336 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068857095744, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 1 without seqno , total = 0 2026-06-08 Mon 06:00:11:318.965 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069376071232, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 2 without seqno , total = 0 2026-06-08 Mon 06:01:11:324.401 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069405460032, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 3 without seqno , total = 0 2026-06-08 Mon 06:02:11:324.934 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068857095744, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 4 without seqno , total = 0 2026-06-08 Mon 06:03:11:325.414 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068861294144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 5 without seqno , total = 0 2026-06-08 Mon 06:04:11:325.900 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068857095744, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 6 without seqno , total = 0 2026-06-08 Mon 06:05:11:326.434 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068865492544, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 7 without seqno , total = 0 2026-06-08 Mon 06:06:11:327.006 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 8 without seqno , total = 0 2026-06-08 Mon 06:07:11:328.340 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068848698944, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 9 without seqno , total = 0 2026-06-08 Mon 06:08:11:329.112 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 10 without seqno , total = 0 2026-06-08 Mon 06:09:11:329.788 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068857095744, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 11 without seqno , total = 0 2026-06-08 Mon 06:10:11:331.753 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068861294144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 12 without seqno , total = 0 2026-06-08 Mon 06:11:11:332.398 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068857095744, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 13 without seqno , total = 0 2026-06-08 Mon 06:12:11:333.367 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068848698944, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 14 without seqno , total = 0 2026-06-08 Mon 06:13:11:333.967 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068865492544, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 15 without seqno , total = 0 2026-06-08 Mon 06:14:11:334.637 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069401261632, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 16 without seqno , total = 0 2026-06-08 Mon 06:14:52:987.556 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:46760 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-08 Mon 06:15:00:429.254 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:54262 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-08 Mon 06:15:11:335.392 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068848698944, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 17 without seqno , total = 0 2026-06-08 Mon 06:16:11:336.757 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 18 without seqno , total = 0 2026-06-08 Mon 06:16:30:659.339 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:47438 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-08 Mon 06:16:52:363.124 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: StartPipeline on : Redis:10.0.0.38 Port:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 812 2026-06-08 Mon 06:16:52:363.700 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: Got Query to prepare for QID a40c8f23-6301-11f1-8be6-00000a000026 src/contrail-analytics/contrail-query-engine/query.cc 1165 2026-06-08 Mon 06:16:52:363.761 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 :0:json query is: { end_time: 1780899412275063 , enqueue_time: 1780899412362573 , query_metadata: {"enqueue_time": 1780899412362439} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1780899394997104 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-08 Mon 06:16:52:363.907 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: Chunks: 3 Need Merge: 0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 936 2026-06-08 Mon 06:16:52:363.922 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: Getting Least Loaded Conn as :0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 986 2026-06-08 Mon 06:16:52:364.000 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: Starting Pipeline for a40c8f23-6301-11f1-8be6-00000a000026 , 1 conn, 16 tasks src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 994 2026-06-08 Mon 06:16:52:365.141 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068857095744, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 :0:json query is: { end_time: 1780899412275063 , enqueue_time: 1780899412362573 , query_metadata: {"enqueue_time": 1780899412362439} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1780899394997104 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-08 Mon 06:16:52:487.950 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:488.024 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:488.033 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:488.039 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:488.045 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:488.050 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:488.056 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:488.061 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:488.081 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:488.087 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:490.355 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 0 and 6 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:490.403 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 6 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:490.412 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 6 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:490.423 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 6 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:490.429 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 6 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:490.437 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 6 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:490.443 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 6 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:490.451 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 6 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:490.457 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 6 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:490.464 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 6 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:490.672 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:490.772 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:490.873 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:490.970 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:491.065 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:491.151 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:493.002 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 37 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:493.104 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 37 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:493.122 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 37 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:493.168 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 37 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:493.184 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 37 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:493.210 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 37 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:493.220 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 37 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:493.242 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 37 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:493.251 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 UNION between tables of sizes 37 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:52:493.272 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QELog: a40c8f23-6301-11f1-8be6-00000a000026 Resulting size of set 37 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:52:493.352 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 :0:json query is: { end_time: 1780899412275063 , enqueue_time: 1780899412362573 , query_metadata: {"enqueue_time": 1780899412362439} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1780899394997104 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-08 Mon 06:16:52:493.503 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:493.601 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:493.691 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:493.781 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:493.865 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:493.959 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:494.072 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:494.192 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:494.280 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:494.365 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:494.443 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:494.524 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:494.618 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:494.720 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:494.805 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:494.885 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:494.974 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:495.059 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:495.158 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:495.263 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:495.342 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:495.419 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:495.500 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:495.588 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:495.690 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:495.774 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:495.864 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:495.958 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:496.071 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:496.203 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:496.306 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:496.385 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:496.460 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:496.550 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:496.648 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:496.730 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:496.810 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: [SYS_INFO]: QEQueryLog: a40c8f23-6301-11f1-8be6-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:52:497.237 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: Will Jsonify #rows 43 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 542 2026-06-08 Mon 06:16:52:497.568 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: Did Jsonify #rows 43 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 551 2026-06-08 Mon 06:16:52:497.740 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: QE Query Result is {"progress":100, "lines":5, "count":43} src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 586 2026-06-08 Mon 06:16:52:497.955 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: Finished: QID a40c8f23-6301-11f1-8be6-00000a000026 Table ObjectVNTable Time(ms) 134 RedisTime(ms) 0 MergeTime(ms) 0 Rows 43 EnQ-delay1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 661 2026-06-08 Mon 06:16:52:497.976 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069405460032, Pid 1]: [SYS_INFO]: QueryPerfInfo: name = cn-jenkins-deploy-platform-ansible-os-5902-1. table = ObjectVNTable query_stats= [ qid = a40c8f23-6301-11f1-8be6-00000a000026 where = [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-60644681:ctest-vn22-14615055", "op": 1, "value2": null}]] select = { "select" : ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] } post = time_span = 8 error = None chunks = 3 chunk_where_time = 123,0, 127,0, 126,0, chunk_select_time = 0,0, 0,3, 0,0, chunk_postproc_time = 0,0, 0,0, 0,0, chunk_merge_time = final_merge_time = 0 time = 134 rows = 43 enq_delay = 1 ] file = src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc line = 657 2026-06-08 Mon 06:16:52:498.047 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: [SYS_INFO]: QELog: Result 1 , 1 conn src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 699 2026-06-08 Mon 06:16:55:491.459 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: StartPipeline on : Redis:10.0.0.38 Port:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 812 2026-06-08 Mon 06:16:55:491.742 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: Got Query to prepare for QID a5e9f4df-6301-11f1-bbc7-00000a000026 src/contrail-analytics/contrail-query-engine/query.cc 1165 2026-06-08 Mon 06:16:55:491.779 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QEQueryLog: a5e9f4df-6301-11f1-bbc7-00000a000026 :0:json query is: { end_time: 1780899415476501 , enqueue_time: 1780899415491082 , query_metadata: {"enqueue_time": 1780899415490992} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1780899394997104 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "2ee79f07-933d-4257-a671-8c4bcb2766a9", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-08 Mon 06:16:55:491.907 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: Chunks: 3 Need Merge: 0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 936 2026-06-08 Mon 06:16:55:491.920 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: Getting Least Loaded Conn as :0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 986 2026-06-08 Mon 06:16:55:491.985 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: [SYS_INFO]: QELog: Starting Pipeline for a5e9f4df-6301-11f1-bbc7-00000a000026 , 1 conn, 16 tasks src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 994 2026-06-08 Mon 06:16:55:492.105 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: [SYS_INFO]: QEQueryLog: a5e9f4df-6301-11f1-bbc7-00000a000026 :0:json query is: { end_time: 1780899415476501 , enqueue_time: 1780899415491082 , query_metadata: {"enqueue_time": 1780899415490992} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1780899394997104 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "2ee79f07-933d-4257-a671-8c4bcb2766a9", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-08 Mon 06:16:55:561.874 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:561.916 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:561.923 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:561.929 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:561.935 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:561.940 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:561.946 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:561.952 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:561.960 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:561.968 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:562.253 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:562.268 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:562.274 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:562.289 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:562.296 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:562.303 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:562.311 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 0 and 3 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:562.329 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 3 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:562.336 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 3 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:562.343 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 3 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:562.383 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QEQueryLog: a5e9f4df-6301-11f1-bbc7-00000a000026 :0:json query is: { end_time: 1780899415476501 , enqueue_time: 1780899415491082 , query_metadata: {"enqueue_time": 1780899415490992} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1780899394997104 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "2ee79f07-933d-4257-a671-8c4bcb2766a9", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-08 Mon 06:16:55:562.493 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QEQueryLog: a5e9f4df-6301-11f1-bbc7-00000a000026 ObjectVMTable :0:Column ObjectTypeName1 Table ObjectVMTable object_id 2ee79f07-933d-4257-a671-8c4bcb2766a9 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:55:562.594 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QEQueryLog: a5e9f4df-6301-11f1-bbc7-00000a000026 ObjectVMTable :0:Column ObjectTypeName1 Table ObjectVMTable object_id 2ee79f07-933d-4257-a671-8c4bcb2766a9 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:55:562.913 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QEQueryLog: a5e9f4df-6301-11f1-bbc7-00000a000026 ObjectVMTable :0:Column ObjectTypeName1 Table ObjectVMTable object_id 2ee79f07-933d-4257-a671-8c4bcb2766a9 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:55:574.371 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 0 and 2 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:574.436 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 2 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:574.445 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 2 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:574.453 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 2 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:574.460 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 2 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:574.467 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 2 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:574.473 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 2 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:574.481 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 2 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:574.489 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 UNION between tables of sizes 2 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-08 Mon 06:16:55:574.498 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: a5e9f4df-6301-11f1-bbc7-00000a000026 Resulting size of set 2 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-08 Mon 06:16:55:574.789 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068857095744, Pid 1]: [SYS_INFO]: QEQueryLog: a5e9f4df-6301-11f1-bbc7-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 2ee79f07-933d-4257-a671-8c4bcb2766a9 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:55:574.929 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068857095744, Pid 1]: [SYS_INFO]: QEQueryLog: a5e9f4df-6301-11f1-bbc7-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 2ee79f07-933d-4257-a671-8c4bcb2766a9 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-08 Mon 06:16:55:575.142 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: Will Jsonify #rows 5 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 542 2026-06-08 Mon 06:16:55:575.245 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: Did Jsonify #rows 5 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 551 2026-06-08 Mon 06:16:55:575.780 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: QE Query Result is {"progress":100, "lines":1, "count":5} src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 586 2026-06-08 Mon 06:16:55:576.000 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: Finished: QID a5e9f4df-6301-11f1-bbc7-00000a000026 Table ObjectVMTable Time(ms) 84 RedisTime(ms) 0 MergeTime(ms) 0 Rows 5 EnQ-delay0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 661 2026-06-08 Mon 06:16:55:576.073 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068865492544, Pid 1]: [SYS_INFO]: QueryPerfInfo: name = cn-jenkins-deploy-platform-ansible-os-5902-1. table = ObjectVMTable query_stats= [ qid = a5e9f4df-6301-11f1-bbc7-00000a000026 where = [[{"name": "ObjectId", "value": "2ee79f07-933d-4257-a671-8c4bcb2766a9", "op": 1, "value2": null}]] select = { "select" : ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] } post = time_span = 8 error = None chunks = 3 chunk_where_time = 70,0, 79,0, 69,0, chunk_select_time = 0,0, 0,0, 0,0, chunk_postproc_time = 0,0, 0,0, 0,0, chunk_merge_time = final_merge_time = 0 time = 84 rows = 5 enq_delay = 0 ] file = src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc line = 657 2026-06-08 Mon 06:16:55:576.087 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: [SYS_INFO]: QELog: Result 1 , 1 conn src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 699 2026-06-08 Mon 06:17:11:337.407 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068861294144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 19 without seqno , total = 0 2026-06-08 Mon 06:18:11:338.352 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 20 without seqno , total = 0 2026-06-08 Mon 06:19:11:338.959 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069392864832, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 21 without seqno , total = 0 2026-06-08 Mon 06:20:11:339.572 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 22 without seqno , total = 0 2026-06-08 Mon 06:21:11:340.297 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069376071232, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 23 without seqno , total = 0 2026-06-08 Mon 06:22:11:340.907 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069380269632, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 24 without seqno , total = 0 2026-06-08 Mon 06:23:11:341.604 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069359281728, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 25 without seqno , total = 0 2026-06-08 Mon 06:24:11:342.188 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069384468032, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 26 without seqno , total = 0 2026-06-08 Mon 06:25:11:342.901 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068857095744, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 27 without seqno , total = 0 2026-06-08 Mon 06:26:11:343.630 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069397063232, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 28 without seqno , total = 0 2026-06-08 Mon 06:27:11:344.282 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069405460032, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 29 without seqno , total = 0 2026-06-08 Mon 06:27:48:073.834 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:42412 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-08 Mon 06:28:11:344.863 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068844500544, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 30 without seqno , total = 0 2026-06-08 Mon 06:29:11:345.819 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140068852897344, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 31 without seqno , total = 0 2026-06-08 Mon 06:29:56:233.019 UTC cn-jenkins-deploy-platform-ansible-os-5902-1 [Thread 140069412599936, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:42520 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256